"New Zealand don't contribute really anything to the defence effort that we've got where we're trying to surveil boats that might be on their way to New Zealand. So I hope that Andrew Little reflects a little more on the relationship between Australia and New Zealand where we do a lot of the heavy lifting. We intercept boats which stop them from making their way to New Zealand.
"There's a lot to this relationship and I was really disappointed in Mr Little's comments during the week. I hope that he doesn't repeat them," Dutton said.
Little said today he wasn't sure which comments Dutton was referring to.
"I think it's the general tone that I've, as Peters has on the issue of the 17-year-old, called out Australia on what looks like a practice that doesn't accord with basic human rights.
"My comment about venality was about the fact that the politics in Australia suggest some widespread attraction to treating people from other countries ... in this sort of this way.
"I think there are plenty of politicians in Australia who hide behind that political calculus and I think that political calculus is venal.
On Dutton's comments that New Zealand didn't do much in the way of regional surveillance, Little said that was a distraction and totally unrelated to the issue of the human rights integrity of deporting people on non-specified character grounds.
"As minister in charge of our two primary intelligence agencies I am totally confident about the contribution we make in the interests of New Zealand, of the region and of the world.
Dutton also expressed frustration at not being able to deport New Zealanders with dual citizenship.
"In limited circumstances, I can strip citizenship from somebody who's been naturalised and the AAT [Administrative Appeals Tribunal] can overturn my decision in that regard. I think it's a stupid situation, I think it's a deficiency in the law. I've tried to get it through the Senate, Labor won't support the change," Dutton told 2GB.
The AAT last week heard an appeal by the 17-year-old New Zealander whose case sparked the exchange between the ministers.
He had been held in an adult immigration detention centre in Melbourne after his visa was revoked. He has been in Australia with his family for around seven years.
He was released this week but his freedom could be short-lived as Dutton can still overrule the AAT's decision.
Dutton is currently taking legal advice on his options but has made it clear the youth will be deported.
"We will make sure that he's deported at the first available opportunity but at the moment he's delaying his return to New Zealand," he said recently.
The youth, who has a string of convictions, is back with his family in New South Wales in the meantime.
Peters also weighed in on the case recently, saying Australia was breaching the UN Convention on the Rights of the Child, and telling it to honour its obligations.