Mr Joyce said a private debt collection agency in Australia would be used to help them with borrowers in default in the country.
"This tracing activity involves a group of 10,000 borrowers, believed to be in Australia, who have made little or no effort to repay their debt, or even make contact with Inland Revenue," said Mr Joyce.
The Government will extend child support border arrest sanctions to non-compliant overseas-based borrowers with high levels of default on their student loan repayments.
Yesterday Pete Hodkinson, NZ University Students' Association president, said the measures were unduly harsh.
"If you look at the rates of repayments for domestic-based graduates, they generally pay back their loans within about five years. It's not like these students once they go overseas suddenly completely change their moral outlook towards their loans.
"The Government should put some more effort into making it easier for students to repay their loans, rather than tacking on more and more obligations ... it's all stick and no carrot."
Labour's tertiary education spokeswoman, Megan Woods, said a heavy-handed approach could hinder the Kiwi OE, where young people gained crucial skills before returning to New Zealand.
"Our concern is that there will be changes that will actually stop an important part of people's career development."
However, Mr Joyce said nearly half a billion of debt was owed by overseas-based student loan borrowers, and the new arrest measure was not harsh.
"It would only be for the people who are deliberately refusing to take any notice.
"We are talking about people that have been contacted by IRD and are refusing to act and then effectively deliberately defaulting."
The Inland Revenue Department will put in place an ongoing information sharing agreement with Internal Affairs to collect contact details from passport applications.
The Government says an overseas borrower with a $20,000 student loan balance will pay their loan off in about 15 years, compared to more than 35 years under the current system. Under new rules the interest they pay would be $8425 - compared with $19,940 under the existing rules.
Mr Joyce said the cost to the taxpayer had reduced from 48c in the dollar in 2009 to 39c in the dollar in March this year.
Revenue Minister Peter Dunne said since IRD's programme to rein in overseas-based borrowers' debt, almost $60 million had been collected. He said previously around 14 per cent of overseas-based borrowers did not meet the interest charged on their loans.
Borrowers say hard line on paying debts will backfire
The Government's increasingly hardline attitude to student loan borrowers will backfire if not matched by efforts to make it easier for repayments, says one London-bound Kiwi.
Software engineer Andy Hayes, 26, has spent two years living in London and is about to return for a further five after collecting a new visa in Auckland today.
Mr Hayes said he gave up repaying his loan after a previous attempt. After battling time differences to get an IRD log-in number, he found his American Express card would not be accepted.
Then, after incurring fees to transfer £5000 ($9200) to a New Zealand account, he was gobsmacked when he was charged a $40 administration cost.
"I just gave up on the idea," said Mr Hayes, who racked up about $35,000 in debt studying computer science at Canterbury University.
He said he was unsure if the Budget changes would cause him to pay down his debt.
"I do worry a tiny bit, just because I do want to come home if I have to ... I've got friends' weddings coming up.
"If the minimums are quite low, I'm happy enough to pay that ... but if they start putting the pressure on and they expect quite a high amount to come back, that's when I probably would say no."
Anna Williams, whose work in the oil and gas industry in Perth enabled her to pay off the remainder of her $59,000 loan last February, said the changes would need to be spelled out to borrowers.
After working during study, the 26-year-old got her loan down to $25,000 before spending a year in the United States.
People going overseas get a "holiday" of one year before they must start making repayments. But Ms Williams said she was unaware she would incur $3000 in interest during that time.
"I can see why they [the Government] want to crack down. But at the same time, it's going to prevent kids who actually do just want to get out and travel. I think if they do bring in higher repayment rates, they've got to spell it out."
Revenue Minister Peter Dunne said making payments from overseas had become easier.
"Inland Revenue has set up toll-free numbers in Australia and the UK and has a list of free online money-transfer service providers on their website."
Student loans
• Overseas-based defaulters could be arrested at the border.
• Debt collection agency employed to trace 10,000 borrowers believed to be in Australia.
• Fixed repayment obligations and higher repayment thresholds introduced.
• People aged 40 and over will be restricted to 120 weeks of student allowances, while people aged 65 and over will no longer be eligible for an allowance.
Loan balance and amount to be paid per year
• • $1000-$15,000: $1000.
• $15,000-$30,000: $2000.
• $30,000-$45,000: $3000.
• $45,000 - $60,000: $4000.
• Greater than $60,000: $5000.
