Read more of the Herald's coverage of this today:
Diplomat will face charges at home, Key predicts
Editorial: Diplomat's nation should be named
But he added: "We understand that some media organisations are seeking to have that suppression order lifted. And I wish you well.
"I can't see any good public policy reason why you'd want to protect someone from publicity given there won't be a trial."
The Herald and other news organisations are challenging the court order. Mr McCully said Government did not plan to join the legal action.
MFAT called in the relevant country's Head of Mission last night to tell them that New Zealand expected the diplomat to face the consequences of his actions.
Prime Minister John Key said the country had reassured MFAT that they were taking the issue very seriously.
"We made it quite clear that we were under no illusion about how seriously New Zealand took the issue, and we expect the person to be held to account," he said.
The diplomat, aged in his 30s, fled the country after being charged with burglary and assault with intent to rape by Wellington police.
He had followed a 21-year-old woman to her home in Brooklyn on May 9, when the alleged assault occurred.
The Ministry for Foreign Affairs formally asked for the diplomat's home country to waive diplomatic immunity, but it declined.
Government said the man could still face charges in his home country.