Judy Steiner, of Mortgage Link Hawke's Bay, said the lending speed limits had been a "big problem" for Hawke's Bay - particularly so for those trying to get on the first rung of the property ladder.
"It's preventing people from buying homes which they can well afford.
"If they could lift the restrictions, that would be so good for Hawke's Bay.
"We need that ... we haven't had the [same] problem of Auckland and Christchurch."
First home buyers in most provincial towns had suffered under the policy, she said.
According to the latest QV figures, the average house value in Napier reached $325,051 last month - up 1.5 per cent from a year ago.
The average house in Hastings was worth $300,924 - up 1.1 per cent - while the average Central Hawke's Bay house value slipped 0.5 per cent to $208,341.
The Reserve Bank introduced loan to value restrictions in October last year, limiting banks' lending to borrowers with less than a 20 per cent deposit to 10 per cent of new lending.
Mr Spencer said without those restrictions annual house price inflation might be 2.5 per cent higher.
Housing supply conditions had also started to improve and, in Auckland, progress was being made in freeing up the supply of buildable land and improving the consent process.
Labour's housing spokesman, Phil Twyford, called on the Government to "fine tune" the restrictions by applying them only to Auckland and Canterbury.
"Why should regional New Zealand continue to be punished for the Government's failure to fix the Auckland housing crisis and get the Canterbury rebuild happening?"
ASB economist Christina Leung said the Reserve Bank would not remove the restrictions unless it was confident the housing market was slowing on a sustained basis and that interest rates would sufficiently contain house prices.
Westpac chief economist Dominick Stephens said it was possible the restrictions' timetable could change if the housing market took off again.
"But in our view that is unlikely to happen, with interest rates rising as they are."
Hawke's Bay Today will be investigating home affordability in our region and ways for entry-level buyers to take their first steps on the property ladder. Watch out for our coverage in the weeks ahead.