"And they pay more taxes per head than any other members of Australian society. So the case for giving them a fair go - or as Australians sometimes put it - giving us a fair suck of the sav, is very strong indeed."
Peters said New Zealand and Australia both had judicial systems that had evolved from the British legal tradition.
"It a tradition which developed the important concepts of natural justice and habeas corpus. These are not concepts we should lose sight of."
Peters and Justice Minister Andrew Little have recently criticised Australia's deportation policies, which Australian would-be leadership contender and former immigration minister Peter Dutton has been unapologetic for.
Peters and Little singled out Dutton's ability to deport Kiwis without charge or trial.
Speaking at a joint press conference with Bishop earlier today, Peters would not directly say what impact a Dutton leadership might have on the deportation policy.
"Whether you're talking about any particular politician ... nobody is beyond is redemption, that's the approach we take from New Zealand," Peters said, to laughter from Bishop.
"With regard to any personality ... be it in my country or your country where our views might differ, our joint approach is to carry on the conversation, improve the dialogue and see a way through.
Asked by a reporter whether that may be more difficult with Dutton as prime minister, Peters said: "I think you're at the wrong press conference."
Peters said the fact that deported people had gone on to commit crimes when back in New Zealand was concerning but those were recent decisions that were a work in progress.
Speaking about a call for New Zealand to take 119 children and their families from Australian detention centres on Nauru, Peters said Prime Minister Jacinda Ardern did not directly approach Nauru.
"The Prime Minister made it very clear that that is the issue for the Nauruans in concert with Australia to decide, not us."
Green Party MP Golriz Ghahraman said now that Dutton had resigned as Australia's immigration minister, it was time to repeat New Zealand's offer to take the refugees.
"New Zealand has repeatedly offered to take refugees from Nauru. I urge Treasurer Scott Morrison, who is temporarily holding the immigration portfolio, to reconsider New Zealand's offer," she said in a statement.
"I trust our Prime Minister will once again reaffirm our offer to accept refugees whilst visiting the region, to place New Zealand on the right side of history on one of the gravest deliberate human rights abuses happening right here in our Pacific neighbourhood.," she said in a statement.
Ardern said earlier this week the offer to take up to 150 refugees from Nauru and/or Manus Island remained.