Mr Key told reporters he would not support this move: "My advice would be [that would be] a step too far, it would be a little bit silly.
"In the end, he obviously feels offended by the comments that have been made ... but history tells you that it's not a very productive step to take."
Dr Smith told the Herald he would "take the Prime Minister's advice on board". He had asked his lawyer for advice because he felt Mr Haynes had deliberately misrepresented his comments.
Some at the meeting said they were given the impression that if they did not pull back on their criticism of irrigation and "dirty dairying" the minister would restructure the organisation through a legislation change.
Mr Haynes told Radio New Zealand: "From my perspective it was very [clear] they were being castigated. It could be construed as political interference - this was about telling Fish and Game to wind their neck in."
Watch: Anne Gibson interviews Nick Smith
Dr Smith said he was "a fan" of Fish and Game and said he had recently approved an increase in its licensing fees. He said his advice that the council "needed to be pro-water quality and not be seen as anti-farming" was honest opinion, not political interference.
He confirmed he wanted changes to the organisation, but said these were related to improvements to the way people could pay their fishing and hunting licensing fees and not to the council's work on freshwater quality.
Notes taken by a Department of Conservation official and released by Dr Smith's office show that the minister questioned Fish and Game's advocacy work.
The notes said: "Big issue - F&G need to work out what they want to be: a statutory body - legislation and a relationship with Government or an NGO?? Statutory monopoly!!"
Fish and Game is a public entity set up under conservation legislation. It receives no taxpayer money and operates independently.
Read more:
*Nick Smith's legal threat over bullying claim
Dr Smith was criticised by Labour, Greens, and National's coalition partner United Future, who defended Fish and Game's work.
Fish and Game Council
*Public entity established in 1987 by Conservation Act.
*Statutory obligations include development of policy to manage sports fishing and hunting resources, and advocacy work.
*Funded by a fee paid on hunting and freshwater fishing licences, and receives no central or local government money.