The restrictions don't affect Kiwis living abroad and buying with overseas income.
Labour housing spokesman Phil Twyford said the banking system recognised non-resident, foreign buyers as a particular risk. "The banks are taking seriously the risk that they identify, while the New Zealand Government has its head in the sand."
Housing Minister Nick Smith said the decision was Westpac's to make. He was not worried about the move or any signal it sent about risks in the housing market. "The evidence is that foreign buyers are a negligible impact on the housing market."
A Westpac spokeswoman said the restrictions would reduce risk.
Bruce Patten, of mortgage brokerage Loan Market, said he expected more banks to follow the pair's lead. In Australia, the introduction of tougher rules for foreign buyers by some banks saw a flood of loan applications to other lenders. One, Bendigo Bank, said the inquiries exceeded its risk appetite.
Patten said most non-resident, overseas-based buyers would take out New Zealand bank loans, unless they paid cash or had somebody here buy the property for them.
Land Information New Zealand (Linz) data shows 474 out 11,955 houses sold between January and March went to non-residents.
Officials warned that there were limitations to the data, and Labour has dismissed it as worthless.
Sam Yin, chief executive of HouGarden which is New Zealand's most popular real estate portal for Chinese-speaking buyers, said the ANZ and Westpac changes would somewhat dampen buying.
"It will affect some people. They will not buy New Zealand properties. But the numbers of Chinese-based people buying property in New Zealand is low," Yin said, citing the Land Information NZ figures.
Chinese-based people wanting to borrow in their own country had problems getting money overseas due to government restrictions there, Yin said.
BNZ and ASB are not announcing any immediate changes on residential mortgage lending to foreign buyers after the Westpac and ANZ moves.
Following those two banks' ban on lending to non-New Zealand residents yesterday, BNZ and ASB - with billions of dollars of home mortgage lending - say they are watching the situation closely.
But they have no immediate changes to unveil, both said.
A BNZ spokeswoman outlined the position.
"It's a situation we're constantly monitoring. We've got no changes which will be announced at the moment. We are watching what the banks are doing," she said.
ASB is in much the same position.
"ASB keeps its credit criteria under review to reflect current market conditions and, to date, has made no change to the home loan lending policy for non-New Zealand residents," an ASB spokeswoman said.
Reserve Bank Governor Graeme Wheeler yesterday also signalled the bank was considering toughening its loan to value ratio limits on lending to rental property investors "towards the end of the year, or before that."
At yesterday's Reserve Bank news conference, Wheeler cited discussions with the Bank of Canada's Governor who said the investor share of property sales in Canada was about 10 per cent. Wheeler contrasted that with the pretty significant share in New Zealand of over 40 per cent, and how it was rising again in Auckland, Hamilton and Tauranga.
A spokeswoman for the Bankers Association said no statement was being issued about the situation which was a commercial business decision taken by two banks.
Watch: The Economy Hub on New Zealand's growing asset bubble:
Last year, Labour sparked an uproar with data showing that up to 40 per cent of Auckland houses were sold to people of Chinese descent, who make up only 10 per cent of the city's population.
The estimate was based on people with Chinese-sounding surnames.
Meanwhile, the Reserve Bank yesterday left the official cash rate at 2.25 per cent, with Governor Graeme Wheeler flagging rising house prices as a risk to financial stability.
He said investors - who account for 46 per cent of Auckland property sales - could soon be targeted by new loan to value lending rules. The loan to value ratio is the amount of a loan compared with the property value.
Last November, the Reserve Bank tightened those rules to rein in Auckland investors, ensuring banks demand 30 per cent deposits for a mortgage secured against an investment property.
Speaking from Fiji, Prime Minister John Key said he did not think the restrictions would have a dramatic effect, and cited Linz data that showed 474 out 11,955 houses sold between January and March went to non-residents.
"Realistically the amount of foreign buyers active in New Zealand is reasonably insignificant at the moment...it will have some impact, but I don't think it will be dramatic."
Mr Key said he had asked banks in the past how much lending was made to foreign buyers with little connection to New Zealand, and had always been told such lending was "very low".
"I do see it as more symbolic than very significant in terms of their balance sheets...but clearly it will have some impact, and clearly they have decided they are going to do it."