"New Zealand don't contribute really anything to the defence effort that we've got where we're trying to surveil boats that might be on their way to New Zealand. So I hope that Andrew Little reflects a little more on the relationship between Australia and New Zealand where we do a lot of the heavy lifting. We intercept boats which stop them from making their way to New Zealand.
"There's a lot to this relationship and I was really disappointed in Mr Little's comments during the week. I hope that he doesn't repeat them," Dutton said.
Assistant Home Affairs Minister Alex Hawke also expressed his displeasure.
"Disappointing to see the NZ Justice Minister fail to use his TV appearance to remind people who are NZ citizens here to obey the laws of Australia. Obeying the law and not committing serious offences is the best way to not be deported," he tweeted.
During a Sky News interview Hawke had a message for Australians living in New Zealand.
"We would, as the Australian Government, say to citizens in New Zealand, 'Well you must obey the law and if you don't obey the law you may be deported back to Australia'.
"We'd like to hear the same message from the New Zealand Government."
Comment is being sought from Little today.
Dutton also expressed his frustration at not being able to deport New Zealanders with dual citizenship.
I was really disappointed in Mr Little's comments during the week. I hope that he doesn't repeat them.
"In limited circumstances, I can strip citizenship from somebody who's been naturalised and the AAT [Administrative Appeals Tribunal] can overturn my decision in that regard. I think it's a stupid situation, I think it's a deficiency in the law. I've tried to get it through the Senate, Labor won't support the change," Dutton said.
"I really want to get this law through."
The AAT last week heard an appeal from a 17-year-old New Zealander who had been held in an adult immigration detention centre in Melbourne after his visa was revoked. He has been in Australia with his family for about seven years.
He was released this week but his freedom could be short-lived as Dutton can still overrule the AAT's decision.
The Department of Home Affairs can also appeal against an AAT decision in court.
Dutton is currently taking legal advice on his options but has made it clear the youth will be deported.
"We will make sure that he's deported at the first available opportunity but at the moment he's delaying his return to New Zealand," he said recently.
The youth, who has a string of convictions, is back with his family in New South Wales in the meantime.
Acting Prime Minister Winston Peters also weighed in on the case recently, saying Australia was breaching the UN Convention on the Rights of the Child and telling it to honour its obligations.
Little told the Herald this week that deportation was the ultimate restriction on liberty.
Every country could make its own rules but they were all part of the global community and some signed up to international agreements, standards of conduct and the rule of law.
"When the state exercises powers that encroach on individual rights, particularly as draconian as removing someone from a country and preventing them from ever returning, you exercise those powers with considerable care and with considerable oversight.
"If that isn't happening in these cases then that is a matter of concern," Little said.
Non-citizens of New Zealand cannot be deported after 10 years of residency here.
Those convicted of non-serious crimes cannot be deported after five years' residency.
