Peter Felstead, a tax partner with Deloitte, said the advice would come as a surprise to many people.
"I think a lot of people will be caught without knowing."
Jewellery, ornaments and collectibles like coins or art are exempt from the tax ruling as they can have another purpose - to be aesthetically pleasing.
But if you are trading those items then you could be liable to pay tax.
Felstead said anyone considering buying gold or silver should consider the tax implications and weigh that decision against other investments.
"It would not be prudent to go investing without considering the tax implications."
Samson Heng, a bullion trader at NZ Mint said, as it had only been mentioned in a draft note it had not taken any action on the advice.
He said taxing the income from selling bullion would be very hard to track.
"It would be very very hard to prove. If you purchase gold you can give it to anyone to sell. It doesn't have your name on it."
He said the IRD would have to set up a registry for each bar which could prove difficult as all of the gold traders in New Zealand were privately owned.
Heng said people often wanted privacy and didn't want to say why they were buying or selling bullion.
Some bought it as an inheritance to give to their children or grandchildren.
He said taxing the proceeds would put people off investing in gold and silver and he questioned the need for it.
"We don't have capital gains on property, so why should we have it on gold?"
An IRD spokesman said it had put out the consultation document to provide some clarity about an issue for which there appeared to be some confusion among taxpayers.
"This is unlikely to be any surprise to tax professionals and financial advisers.
"However, it is possible investors who have not sought advice may not be aware of the tax implications of investing in gold.
"We hope that the consultation process will provide greater clarity for investors in gold bullion and as such they will be more aware of their tax obligations.
Felstead said affected investors should make a submission on the IRD's draft document.
Submissions close on April 7.
He said it typically took the IRD around eight weeks to respond to the submissions and give its final view but it could take longer if there were a lot of submissions.