Police are investigating the illegal purchases and have warned firearms dealers, gun clubs, collectors and other sellers about the forged licences.
They released an image of the licence allegedly used in the operation.
"Police are interested in hearing from anyone who may have been approached by him using these identities, or who may have come across anyone acting suspiciously while trying to purchase firearms - and we encourage dealers, collectors or members of the public selling firearms legitimately to remain vigilant," detective sergeant Glenn Toy of Wanganui Police said.
If dealers or anyone else selling firearms has concerns about the legitimacy of someone who may approach them to buy firearms, they should contact police immediately.
"While at this stage there is nothing to suggest any wider threat to the public, we take any such illegal purchase seriously and will respond accordingly to ensure public safety."
Yates told the Herald on Sunday every firearms dealer in New Zealand would have the mail order forms on their website. They contain the dealers' name, licence number and expiry date.
They are used to apply for a permit to buy a pistol, restricted weapon or military-style semi-automatic firearm from police.
Once the permit is granted by police, a distinctive pink form is given to the buyer to fill out, and a carbon-copy is given to the person selling the gun to fill in.
However, Yates believed the alleged forger had simply presented the application downloaded from his website to buy the guns, rather than using the pink form.
"Anyone who is a collector should know - because they'd have presented, for however many guns they've bought - the two-part pink document," he said.
A police spokesman said it was "early days" in the investigation, but the potential issue of the collector selling the gun without the correct paperwork would be looked into.
Firearms safety specialist Nicole McKee believed the procurement permit had been forged.
McKee said the incident was extremely concerning for licensed firearms owners.
Last year Story presenter Heather du Plessis-Allan bought a gun to expose loopholes in how guns can be bought online.
It caused a furore but police eventually decided against charging the presenter and her team.