"I'm slightly surprised that after such a big issue that they would taper so soon," Westpac senior markets strategist Imre Speizer said, adding the move had prompted some selling in the market.
"Tapering is a big thing from the market's point of view and it's a reasonable step down," he said.
Similar bond tapering moves have been made by the US Federal Reserve.
On Thursday, the Financial Times reported that the Bank of England voted to pump an additional £100bn into the UK economy on Thursday but, with financial markets more stable than in March, it felt able to slash the pace at which it would inject the money.
The Reserve Bank of Australia has stopped buying bonds altogether.
"Better performing bond markets and improved market functioning have likely been factors at play in the RBNZ's decision to taper, as has the global trend towards tapering," ANZ strategist David Croy said.
"At the margin, it is one of the themes that is coming through in the market right now," Croy said.
The Reserve Bank has set a cap of $60b for its bond-buying but ANZ expects the central bank to raise that to $90 billion in August, based on its expectation that the economy will require extra support.
The central bank is due to review its official cash rate next Wednesday and market expectations are for Reserve Bank Governor Adrian Orr to leave it unchanged at 0.25 per cent.
Westpac expects the bond-buying cap to be lifted at some point, it said.
"There is a chance that the cap will rise to $70bn next week. But more likely, the RBNZ leaves the cap unchanged this time, and lifts it to $80bn in August in a single hit," Westpac said in a commentary.
"The RBNZ is likely to reiterate that it has other monetary tools available, but will not signal that these are necessary at this point."
