Nauru bit back via its Government Information Office Twitter account, saying the decision to suspend aid was "misguided & based on misinformation" and that Mr Kun was under investigation for his part in a "riot" outside Parliament. The action taken by New Zealand while that was underway "seeks to undermine our national sovereignty & influence a criminal investigation."
The Twitter account said the protest for which several now faced trial was a "riot" aimed at overthrowing a democratically elected Government "and should be condemned by all who value democracy."
"Govt of Nauru implicitly respects the rule of law and the separation of powers, however we also stand by our right to uphold the law."
It also defended the decision to replace judges, saying it was part of President Baron Waqa's reforms to "end corruption and cronyism of the past."
Mr McCully said he was assured by Nauru's Justice Minister David Adeang last year that the government would work to restore confidence in the justice system but that had not happened.
"To put the funding on hold for any justice sector support is something we do very reluctantly but we are in a position where we think that support is going to be viewed as part of the problem rather than part of the solution if we allow things to go forward without any shift in approach."
The Minister said there was concern about the need for those at the asylum seekers' centre to have access to justice.
"That remains a concern of ours and it remains a reason for us to work to resolve the differences."
Mr McCully said one of the main concerns he raised with Mr Adeang was the case of opposition MP Roland Kun whose family is now in New Zealand. Mr Kun's passport was seized and cancelled on a visit to Nauru and he is unable to return to his family. Mr McCully said the decision to seize the passport was made by the Justice Minister rather than the courts which was concerning.
"There have been a range of steps taken there that you wouldn't normally see in a place where there was a proper separation of powers and where the rule of law was being pursued in the way we would recognise in New Zealand."
The funding was on hold until New Zealand was satisfied the appropriate steps were taken to restore confidence in the justice sector.
"I did talk to Mr Adeang a couple of days ago, we haven't managed to resolve things. For that reason we will leave things in a holding pattern until we can."
Although the annual $1.2 million was not a significant amount, it was a large chunk of the justice sector funding for Nauru and New Zealand was the only country which put aid directly into the justice department.
Mr McCully said he had not yet spoken to his Australian counterpart Julie Bishop but expected to in the next few days.
There has been criticism of Australia's refusal to take a hard line and claims it was compromised by the asylum seeker centre it has in Nauru. Mr McCully had earlier expressed hopes of a united stance but today was reluctant to criticise Australia. He said Australia had a different relationship with Nauru.
"We were the unfortunate ones here who were funding the very justice system that has been the subject of international criticism. I believe Australia's situation is totally different."
Mr McCully expected to hold more talks with Nauru when in Papua New Guinea for the Pacific Islands Forum next week.
In July Mr Kun's case was raised in a letter by 29 legal academics in New Zealand who called on the Government to remove aid from the justice sector if proper rule of law was not restored.
Otago University law lecturer Andrew Geddis says he is pleased the New Zealand Government has suspended its funding to the Nauru justice department, saying it was a step that had to be taken.
Mr Geddis was one of the 29 legal academics who wrote to the Government urging it to cease its funding of Nauru's justice department unless the Nauru government took measures to address a breakdown in the rule of law and civil rights abuses.
Speaking on behalf of the signatories, Mr Geddis said it was disappointing that matters were not resolved to allow the funding to continue.
"But the Government has taken the right steps given there has been no significant improvement in the situation in Nauru. It is essentially without a properly functioning judicial system and New Zealand should not be complicit in aiding that."