Horne was caught after his adoptive parents reported him to the police. Horne pleaded guilty to kidnapping and indecent assault on a female under 12.
In September 2014, he was sentenced to three years in prison, due to end in January 2017.
A deportation liability notice was served on him on March 19, 2015, and in April Horne lodged an appeal against deportation with the Immigration and Protection Tribunal.
In a decision released in April this year, the tribunal suspended the deportation liability due to "exceptional humanitarian circumstances".
The tribunal found Horne would have no family and support in South Africa unless his father moved back with him, but doing so would cause a "permanent and irretrievable breakdown of the family unit", as Horne's mother and siblings would stay in New Zealand so as to not disrupt the siblings' education.
"There is not one member, in what is a close-knit immigrant family, who will not be adversely affected by the appellant's deportation," the tribunal decision said.
That, combined with the fact Horne has little to no connection with South Africa and is dependant on his family for emotional support and communication, made deportation "unjust or unduly harsh".
In a pre-sentence report in 2014, Horne was found to be at a moderate to high risk of reoffending, but by May 2015, a clinical psychologist determined him to be in the moderate to low range.
Since then, Horne has begun sex offending counselling.
The tribunal also took into account Horne's parents, who it considered to be "responsible members of the public".
This was evident in their decision to turn their son in, as well as the decision to move to a different suburb so they were not living near the victim.
"The couple have shown considerable forethought and planning in devising a safety plan for the appellant and managing his reintegration into the community and his post-custodial rehabilitation. This helps to provide some assurance that the risk of the appellant re-offending will continue to fall."
The tribunal was satisfied it would not be against public interest to allow Horne to stay in the country.