Mr Spencer said without the restrictions, annual house price inflation might be 2.5 per cent higher.
Housing supply conditions was improving and in Auckland progress was being made in freeing up the supply of buildable land and improving the consent process.
"The financial system is less vulnerable to an adverse housing shock and banks are now less exposed to potential credit losses as the interest rate cycle turns upwards."
At this stage, he said, the earliest date for beginning to remove the restrictions was likely to be late in the year. While removing the 10 per cent "speed limit" could improve access for first home buyers, it would be just one obstacle removed, said Associate Professor David Tripe, associate head of Massey University's School of Economics and Finance.
"Banks may continue to be somewhat cautious in their lending - one of the things about those rules having been in place is they have cemented a culture of caution. But what has also happened is the banks, in most cases, have not had enough demand for the high LVR loans to be able to even get up to the percentage cap."
Associate Professor Tripe expected that house prices would continue to rise - something that could only be slowed by either a significant economic downturn or a sudden turn in migration trends.
"I suspect that taking [restrictions] off wouldn't necessarily turn the market greatly."
ASB economist Christina Leung said the Reserve Bank timing was in line with its expectations that the restrictions would most likely be removed over a window from late-2014 through to mid-2015.
"The RBNZ will not remove the restrictions unless it is confident that the housing market is slowing on a sustained basis and that interest rates will be sufficient to continue containing house prices, as well as confirmation that net migration is slowing."
Dominick Stephens, chief economist for Westpac, warned it was possible the timetable for restrictions could change if the housing market took off again. "In our view that is unlikely to happen, with interest rates rising as they are."
Real Estate Institute of New Zealand chief executive Helen O'Sullivan said the lending restrictions had "missed the mark" somewhat.
"In our data there has been more of an impact at the bottom end of the market in terms of knocking out transactions, than slowing down what people are prepared to pay at the upper end of the market," she said. "But more than anything else, it has slowed down activity in the market, and it was always our thinking that the volumes would show the greater impact."
The impact of the LVR had been felt in provincial centres, she said, where she didn't feel there was the same supply and demand imbalance seen in Auckland.