Former students in serious default can now be arrested while trying to leave the country or returning from overseas.
New figures showed six warrants for arrest had been issued this year and three people had been arrested at the border - one more than previously reported.
Speaking to reporters at Parliament, Joyce said after the latest arrest was made as the man tried leave New Zealand for Australia. They made arrangements to repay their loan and were allowed to leave the country.
Joyce did not have any detail about them or how much they owed. It is understood that he was travelling with his wife and family at the time of his arrest.
In June, a woman who was in default on her loan was arrested at Auckland Airport as she tried to board a plane to Australia.
And in January, Cook Islands man Ngatokotoru Puna, 40, was arrested as he tried to leave New Zealand - the first time the hardline arrest powers were used since being introduced in March 2014.
Opponents of the arrest powers say educated expats will be too afraid to return to New Zealand.
Joyce said there was "no doubt" people were staying offshore to avoid paying off debt.
"What this will make clear to them is they actually need to get on and pay back their student loans so we can use that money for other students."
The total amount of borrowing in default is now $1.07 billion, of which 90 per cent is owed by people living overseas.
Many of them are in Australia, and today an information exchange agreement between New Zealand and Australian tax officials came into force.
The Australian Tax Office identified a total of 10,400 student loan borrowers living across the ditch - around 10 per cent of the total borrowers living abroad.
"Inland Revenue will now analyse the information, identify those who are in default and start contacting them to resume repaying their loans," Joyce said.
He advised them to get in touch with the IRD to work out a repayment plan.
"If they don't, it's likely someone will be knocking on their door in the next few months."
The joint NZ-Australia agreement was a "huge boost" in helping tax officials get in touch with borrowers who have been hard to find or were deliberately avoiding repayments, Joyce said.
IRD were now able to get up-to-date address and contact details for borrowers, and had powers to send debt collectors to their homes.
Revenue Minister Michael Woodhouse said that since changes were made to target "problematic" borrowers, $300 million in debt had been collected.
The IRD was reporting a "turn-around" in attitudes by borrowers who had previously been reluctant to engage, he said.
"It seems the message is getting through that if you leave the country for a few years, you can't leave your student loan behind and hope it just goes away."
