New inflation data shows prices rose even more than many expected in the June quarter.
The Consumer Price Index was up 3.3 per cent year - the market had forecast 2.7 per cent.
This was the biggest increase in nearly 10 years and was driven by higher prices for new housing and petrol, StatsNZ said.
"Previously, annual inflation had peaked at 5.3 per cent in June 2011, impacted by a GST increase of 2.5 percentage points that came into effect in October 2010".
"Before this, annual inflation reached 5.1 per cent in the September 2008 quarter, during the time of the global financial crisis".
The spike takes annual inflation outside the Reserve Bank's 1-3 per cent target range and is likely to put more upward pressure on interest rates.
Market pricing suggests there is now a 90 per cent chance of a rate hike next month.
The official cash rate currently sits at 0.25 per cent.
Banks started lifting mortgage rates this week.
Most economists now expect the Official Cash Rate will be lifted in August.
ANZ economists described the result as "monstrous".
"It's clear that inflation momentum is building rapidly," said senior economist Miles Workman.
"The extremely tight labour market and shift in pricing behaviour by firms mean that the RBNZ can't afford to wait much longer–these factors will only help inflation snowball from here".
The Consumer Price Index (CPI) is a measure that pulls together a weighted average of prices for a basket of consumer goods and services, such as transportation, food and medical care.
It doesn't include house price inflation - which for the past few years has run far hotter than most goods and services - but it does include building costs and rental costs.
The cost of building a new house was the biggest contributor to both annual and quarterly inflation this quarter, up 7.4 per cent for the year, and 4.6 per cent for the quarter.
"Higher prices for building houses reflect both supply-chain problems and high demand," said StatsNZ's prices senior manager Aaron Beck.
"Several construction firms have told us that it is hard to get many of the materials they need to build a house, and that there are higher labour and administration costs.
"In addition, in the year to May 2021, $18.3 billion of residential building work was consented, up 18 per cent on a year earlier."
Rent prices increased 2.9 per cent for the year, and 0.9 per cent for the quarter. In the June 2021 quarter, rent prices went up 0.5 per cent in Auckland, 1.3 per cent in Wellington, and 0.6 per cent in Canterbury.
Petrol prices increased 16 per cent between the June 2020 quarter and June 2021 quarter.
The weighted average price of a litre of 91 octane was $2.13 in the June 2021 quarter, up from $2.00 in the March 2021 quarter.
By the end of the quarter, prices were about 2.6 per cent higher than the average over the quarter.
The New Zealand dollar rallied by about half a per cent to US70.13c on the news.
Wholesale interest rates also firmed.
"It's a decent upside surprise compared to expectations," Hamish Pepper, fixed income and currency strategist at Harbour Asset Management, said.
"It further encourages the market to price in its August 18 meeting as the time when the Reserve Bank will begin its tightening cycle," Pepper said.
Yields in short end of the swaps curve - which has a key influence on mortgage rates - all gained after the data's release.
The one-year swap rate rose by about five basis points to 0.80 per cent, the two years gained seven to 1.11 per cent and the five years also by seven basis points to 1.54 per cent.
In the long end, the 10-year swap rate gained five basis points to 1.54 per cent.
- additional reporting Jamie Gray
New employment data this week will reveal the impact on jobs.