Peters and Justice Minister Andrew Little have criticised Australia's deportation policies, which would-be leadership contender and former immigration minister Peter Dutton has been unapologetic for.
Dutton has resigned as a minister and former Immigration Minister Scott Morrison is acting in the role.
There has been speculation that without Dutton in the immigration portfolio there may be a softening in Australia's hardline stance.
Australian refugee activists said Morrison's appointment was a "monumental step backwards".
"When Immigration Minister for Tony Abbott, Scott Morrison had a track record of removing mobile phones from people in Immigration detention. These phones are a legal as well as emotional lifeline.
"The behaviours of criminals and of Australian asylum-seekers are being deliberately blurred by Home Affairs. This must stop," activist Jane Salmon said in a statement.
A spokesman for Morrison today said he had no comment to make on any possible changes to immigration policies.
A reshuffle is likely after Dutton's resignation and it is not clear whether Morrison would remain in the role.
Speaking at a joint press conference with Bishop earlier today, Peters would not directly say what impact a Dutton leadership might have on Australia's deportation policies.
"With regard to any personality ... be it in my country or your country where our views might differ, our joint approach is to carry on the conversation, improve the dialogue and see a way through."
Speaking about New Zealand's offer to take refugees from Nauru or Manus, Peters said Prime Minister Jacinda Ardern did not directly approach Nauru.
"The Prime Minister made it very clear that that is the issue for the Nauruans in concert with Australia to decide, not us."
Green Party MP Golriz Ghahraman said with Dutton on the back bench, Ardern should repeat the offer.
"New Zealand has repeatedly offered to take refugees from Nauru. I urge Treasurer Scott Morrison, who is temporarily holding the immigration portfolio, to reconsider New Zealand's offer," she said in a statement.
Speaking after a meeting with Polish President Andrzej Duda today, Ardern said she had met World Vision after its call for New Zealand to take 119 children and their families from Nauru.
She said she had reiterated that New Zealand remained ready and willing to take 150 people from Nauru or Manus.
But Australia played a critical role in that and New Zealand would not go directly to Nauru.
"Because of the processing, the nature of the current situation ... New Zealand would need to work alongside Australia and Nauru in order to successfully resettle," she said.