The third attempted victim was an 18-month-old baby. In October 2012, Ellmers tried to pay $500 in return for "sexual gratification" with the child during a trip to Christchurch.
Ellmers was remanded in custody today.
His defence lawyer Shannon-Leigh Litt asked for name suppression and that the matter remain in the district court jurisdiction, but both requests were denied by Judge Bridget Mackintosh.
Crown prosecutor Steve Manning said it was at the "absolute highest end" of child sexual offending in the country, and he successfully argued for sentencing to be moved to the High Court at Hastings in May.
Only the High Court can impose a sentence of preventive detention. Ellmers could be sentenced to a minimum term, and be released thereafter only if authorities are satisfied he no longer represents a significant risk to the public.
Ellmers has a history of such offending in Australia, where he lived from 1999 to 2008. In 2005 he was sentenced to five years imprisonment for rape, attempted rape and indecent treatment of a child under 12. Ellmers was aged 33 at the time. In June 2008 he was deported back to New Zealand after serving 18 months of his sentence.
Two other New Zealand men, in Canterbury and Auckland, also face several charges including sexual violation of children and making, possessing and distributing child abuse images, police said.
The arrests follow a police operation targeting alleged paedophiles in New Zealand and overseas which began in July last year and involved staff across the country including Northland, Auckland, Eastern, Canterbury and Southern districts.
As a result, four New Zealand children, aged 13 months to 13 years, had been saved from further abuse and police had made 35 referrals to authorities in Australia, the United Kingdom, USA and Canada, said Detective Senior Sergeant John Michael, head of the specialist police team fighting child exploitation, OCEANZ, which is part of the National Criminal Investigations Group.
"This offending is like a spider web which has got wider and wider, involving offenders both here and overseas. Those facing charges have allegedly abused children, recorded that abuse and distributed it amongst themselves and overseas."
The nature and depravity of this man's offending has "wreaked havoc on so many lives", said Detective Senior Sergeant David Greig, head of the Eastern District child protection team.
Mr Michael said police were continuing to build their ability to infiltrate such offending in New Zealand and overseas.
"Child victims almost always know their offenders, who have gained their trust and that of their parents, before the offending occurs. We urge parents and caregivers to be alert to the signs of this kind of abuse, but not alarmed," he said.
He urged parents who have concerns about an adult's behaviour or find disturbing images on any computers to contact the police or Crimestoppers anonymously on 0800 555 111.