"Corrections have already done so and that, I think, is entirely fitting," Ardern said from Tuvalu, where she is at the Pacific Islands Forum.
"No one wants extra trauma for the victims, extra trauma for Christchurch, and ultimately no one wants this alleged offender to be able to spread his message of hate in the way he's been able to.
"Now we have to do everything we can to prevent this thing happening again."
National Party Corrections spokesman David Bennett said that Davis should have sought assurances that Corrections was doing all it could to keep the public safe.
"A good Minister would have asked from day one what was happening around correspondence, visits and prison status. This should have been monitored weekly by the Minister so Corrections knew how seriously it should have been taking this.
"Five letters is not a one off mistake. It is incompetence. A Minister is responsible for making sure their department is taking all necessary steps."
Corrections was checking its systems, and Ardern said that was appropriate.
"Clearly there's been a systematic failure here. We need to make sure the department is going back an checking their systems, and equally that we're checking the law.
"If there are any holes here that need to be fixed, that needs to be rectified immediately. This will be an individual who continues to try and spread his hate and that needs to be stopped."
But Bennett said the fault was not with the law.
"The reality is, there are already laws in place which should have been used to stop this from happening.
"Mr Davis now needs to ensure there is help and support in place for the victims of the shootings who may have been re-traumatised by this news."
Last night Ardern said that, ultimately, the person most responsible was the alleged offender.
"No one's going to let anyone off the hook here. This shouldn't have happened, but ultimately that's where the buck needs to stop."