"The operation of the justice system is pretty important to those associated with the asylum seekers' centre - the operation of courts is essential to overseeing everyone's behaviour there. If people don't have courts to appeal to it's a pretty serious matter. So we do have some responsibilities to make sure the justice system is maintained. I know there are those who say 'just pull your funding'. It's not that simple and we shouldn't behave that way with a Pacific neighbour either."
He said he wanted more information from Nauru, especially around actions taken around a protest outside Parliament a fortnight ago in which several people were arrested, including three Opposition MPs who were suspended from Nauru's Parliament in 2014.
"We obviously want to make sure the justice system as we would see it is being managed appropriately in dealing with the inquiries and dealing with any prosecutions."
Mr McCully has also asked for a meeting with Nauru's justice minister David Adeang, who this week criticised New Zealand telling AAP it was "making rash statements without understanding the facts."
Mr McCully said he met Mr Adeang last year after the expulsion of two judges from Nauru.
"I received his assurance that Nauru was committed to rebuilding international confidence in the justice system. And it is for that reason I want to hear from him about recent events and encourage him to set a path that is going to rebuild international confidence."
Mr McCully was optimistic New Zealand and Australia will take a united stance on Nauru despite Australia's luke-warm reaction to the prospect of firm action.
Australia funds the asylum seeker centre in Nauru and many commentators believe it is conflicted in its ability to respond to the concerns.
Australia's foreign minister Julie Bishop has steered clear talk of particular action against Nauru, telling the ABC she had raised concerns with President Waqa but described Nauru's problems as "domestic issues".
Mr McCully said he had discussed it with Ms Bishop.
"I'm hopeful we will be joined up on this and I get a sense that is the case. But I acknowledge, in saying that, that they have very much larger and more complex interests in their relationship with Nauru."
Mr McCully said he had also asked President Waqa about the decision to remove Opposition MP Roland Kun's passport, meaning he could not return to his family in New Zealand.
He was waiting for more information about the processes used in that decision but said the discussion was "robust".
Mr Kun's case was raised in a letter from 29 legal academics in New Zealand who called on the Government to remove aid from the justice sector if proper rule of law was not restored.
Victoria University's chair of public law Claudia Geiringer, said it was encouraging New Zealand's aid relationship had been put directly to President Waqa by Mr McCully.
"In the end, though, my concern is with results - a swift improvement in conditions on the ground in Nauru, including the return of Mr Kun to his family. Or, if not, the withdrawal of aid to the Nauru justice sector."
Mr McCully said President Waqa had disagreed with some of the international reporting especially around the protest.
"It's fair to say Nauru believe international reporting has not taken account of the events surrounding the protest outside Parliament and what they regard as genuine acts of lawlessness by some, and argues authorities in any country would want to pursue those sorts of incidents."
