The move would not include soft drinks, snacks and luxury foods.
"We don't think highly sugared and bad-for-you fizzy drinks are something we should be supporting."
While Labour's policy, abandoned months ago, would have cost $250 million a year, Mr Peters said his policy would save New Zealanders but cost the Crown a whopping $3 billion a year or thereabouts.
"This bold policy aims at the heart of the inequality undermining our society."
Also "as part of a fair system" NZ First would remove GST from rates on residential property.
"This tax-on-a-tax deceit has to end, and it will," Mr Peters told around 150 party faithful at Alexandra Park.
He did not provide details on how much that policy would cost, but with local authorities raising more than $7 billion a year in rates, the Crown would lose hundreds of millions of dollars in revenue.
However, in an echo of Labour's plan to fund its big-ticket items, Mr Peters said the policies would be funded through "a clampdown on tax evasion and the black economy" which he estimated was worth $7 billion a year.
He provided little detail on which areas of tax avoidance the Crown would target other than cash jobs by tradespeople.
However, he referred to his record in taking on tax dodgers, including his crusade over the "Winebox" tax dodge more than two decades ago.
Taxpayers Union executive director Jordan Williams said he welcomed Mr Peters' recognition that New Zealand families were over-taxed, "but introducing new complexity to GST won't reduce the burden".
Local Government NZ president Lawrence Yule said his organisation supported the removal of GST from rates "in principle", but it would be complex "because rates are made up of a combination of both tax and other charges, which are different for every local authority in New Zealand".