"I think there is quite a solid argument to say that while she was born in New Zealand, she has lived her life there, and she has been a product of the Australian system," Mr Key said.
"I think you've kind of got to take a bit of the rough with the smooth. And we do that over here. We do deport the odd person, but after a while we say 'you are fundamentally a New Zealander and you take the good with the bad'."
Deportees are under no obligation to engage with police when they arrive in New Zealand.
Justice Minister Amy Adams, who announced an information-sharing agreement with Australia on deported criminals last month, said the next step was to change the law so deportees were subject to the same conditions they would be had their sentence been served in New Zealand.
Yesterday, Ms Adams said supervision orders could be put in place for the most serious offenders, and the law would soon be changed to cover all deportees.
Colt's case comes only days after Mr Key met counterpart Malcolm Turnbull in Auckland to protest a new law that has seen non-Australians deported if they incur a jail sentence of 12 months or more.
Labour leader Andrew Little said the Colt deportation - if confirmed - was a classic example of what New Zealand was facing because of Australia's tough new law.
"The [New Zealand] Government knew about the law change at the time ... the Government has done nothing about it until now."
Colt was part of a 40-strong group that lived in squalid conditions near Canberra.
In 2012, authorities took away 12 children and testing revealed that only one had parents that were not related.
The children, who were underfed, hardly went to school and lacked basic skills such as how to shower and clean their teeth, later told of suffering sexual abuse.
