Peters said New Zealand had been "pretty lax" and "pretty slack" about gun laws in the past.
"All of a sudden, a day of reckoning has come and we have got to act as we should have done a long time ago."
He said the Port Arthur massacre in Australia in 1996 "should have been a signal to us. We didn't act then, either".
Peters said since no action had been taken for two decades, the Government does not know how widespread the potential illegality, when the new law is passed, will be.
At the moment, it is unknown how many firearms there are in New Zealand but the number is estimated to range between 1.2 million and 1.5 million.
There are 245,000 firearms licences in the country.
The Arms (Prohibited Firearms, Magazines and Parts) Amendment Bill has been introduced to Parliament and will have its first reading today.
Yesterday, Peters told media the funding for the buyback would come from outside existing department baselines.
"It's going to cost us a penny, but that's what has to be done now," Peters said.
"We're going to have to find either savings somewhere else or increased revenue, but whatever it is, that is not the point. It is to treat people fairly who thought they had a legal entitlement to a weapon that's now going to be outlawed."
He also said the amnesty, whereby people can hand back any guns the Government has now deemed to be illegal without being charged, would last until the end of September.
So far, 210 guns have been handed back – Peters told RNZ he expected that number to rise in the coming months.
