Speaking to reporters at Parliament, Corrections Minister Kelvin Davis declined to divulge any further details due to "operational security reasons".
"The cost is what the cost is," Davis said.
Act leader David Seymour said the cost was "completely unjustifiable".
"Five thousand dollars a day is a lot of bread and water. I'd rather see us economise on that ***holes's accommodation."
NZ First leader Winston Peters agreed the figure was "mind-boggling".
"I cannot think it's justified," he said.
Peters told reporters he would like to see Tarrant sent back to his home country of Australia to serve his full sentence.
"If there's an agreement between the Australian Government and New Zealand Government, it's very simple to do," Peters said.
"It would be the honourable thing [for Australia] to do."
Speaking to reporters, Ardern kept open the possibility that the gunman could be sent to Australia, but noted that prisoners typically served their sentence in the country where they committed a crime.
Ardern said the victims' desires would also be a determining factor.
"We've never taken it off the table, but we do need to make sure that justice is done here first. We haven't had the sentencing yet. We need to see that through."
National leader Judith Collins said "the rule of law" stated that Tarrant must and should serve his sentence in New Zealand.
"He committed his evil crime here. I think he needs to have his sentence here, even though it costs us a lot of money," she said.
"I am not at all comfortable with us paying for anything for that man, but that is the way it is and we have to front up."
Collins pointed out that hundreds of New Zealanders were currently serving sentences in Australia.
Tarrant's sentencing is scheduled to begin later this month. He's expected to spend the rest of his life behind bars, whatever country that may be in.
- RNZ
