Mr Key said the Government "expressed [its] view" that the man should have faced the charges in New Zealand.
"But it is, as you know, up to the home country to decide whether they invoke diplomatic immunity or not."
Police began criminal proceedings against the man last month and notified the Ministry of Foreign Affairs and Trade (MFAT).
MFAT asked his host country to waive immunity, but that country's government refused, and pulled him from his posting in Wellington.
The ministry sent a diplomatic note to his home country to say it wanted the man to face the sexual assault charges in New Zealand.
Mr Key said he understood an investigation was underway in the diplomat's country.
"It's an ongoing process here at the moment, there's a suppression order so I can't go into too much detail. But my understanding is that the home country is looking at the case and charges might follow there."
The police file remained alive and if the diplomat returned to New Zealand in a non-diplomatic position he could be arrested.
Labour leader David Cunliffe said the public had a right to know which country the diplomat was from.
"There are well established conventions about diplomatic immunity which are probably beyond the control of any New Zealand Government, but ... we should know where he was from," Mr Cunliffe said.
He said he did not know the nationality of the diplomat.
Labour's foreign affairs spokesman David Shearer said Foreign Affairs Minister Murray McCully needed to ensure the matter was not "swept under the carpet and forgotten about.
"The alleged victim in the case deserves more than that," he said.
Mr Shearer said extradition of the diplomat to face charges in New Zealand should be considered.
He also questioned the name suppression for the diplomat and his home country.
"The Vienna Convention that provides diplomatic immunity is an ancient agreement, one that New Zealand takes seriously and upholds. It protects our diplomatic personnel serving overseas, no matter where they are located.
"Nevertheless, we also expect justice to be done and the rights of any victim to be respected and not simply forgotten."
Police told the Herald on Sunday they had sufficient evidence to charge him with assault with intent to rape, but had let him go as he was not able to be prosecuted under New Zealand law. He had also been charged with burglary.
It is understood the man refused to give a DNA sample.
Police spokesman Nick Bohm said the complainant "has been fully informed throughout the process and we are continuing to support her".
MFAT said serious crimes by envoys in New Zealand were rare, but Louise Nicholas, survivor advocate for Rape Prevention Education, has labelled the diplomatic immunity a "crime in itself".
"It's really difficult for the Government. There is nothing you can do about it. The victim has to live with knowing there will be no justice for her. It is disgusting of the [diplomat's] government to allow this man home.
"For these people to be able to legally walk free is a crime in itself."
Nicholas praised the NZ Government's attempts to get the man's immunity waived, and still wants him to be extradited.
"The crime was committed here so he should be brought back here to face the judicial process ... It's just so wrong."
- additional reporting by Patrice Dougan
