In fact, one doctor, Dr Nina Sawicki, said she has quit the association because of its stance.
"I decided to leave because I felt that their stance on the cannabis referendum, while it wasn't incorrect, I felt it was incomplete," Sawicki said.
"I didn't feel that it addressed the wider spectrum of harm that comes from the current use of cannabis. I felt they took a very narrow medical focus on the harm of cannabis."
She said the NZMA is not representative of the entire medical workforce in this country, and that many doctors have opposing views.
Another doctor supporting legalisation and denouncing the NZMA's stance is Dr Richard Medlicott, the medical director at the Royal New Zealand College of General Practitioners.
"I think if you look at the evidence around use in New Zealand which can be harmful, this legislation is well written, puts in protections, and I expect will reduce harm."
Opponents to legalisation say some of those provisions can be put in place without legalising weed, such as better access to health services.
The NZMA said civil penalties should be in place for cannabis possession, leading to counselling, education, and treatment, rather than pushing people through the criminal justice system.
It would be a form of decriminalisation, rather than legalisation.
"It's kind of a change in the rules that doesn't really go far enough," Medlicott said.
"You'd still have people that want to purchase marijuana going to some pretty dodgy places and mixing with some pretty dodgy characters.
"You'll still have high-potency marijuana available, you'll still have it being pushed on the under-20s.
"I don't think decriminalisation is the way, I think legalisation has to be the better option."
Dr Stephen Graham, the medical director at WellSouth in Dunedin, supports legalisation and said it will take away the anxiety some people feel when needing help regarding cannabis use.
He said doctors need to be more honest about whether patients are honest about cannabis habits.
"I don't think we know if people are being open with us about it," Graham said. "We really can't say.
"There are still the legal concerns in there and it does get in the way of open discussion."
That was echoed by Dr Sawicki.
"It's hard enough getting an accurate alcohol history from patients, and alcohol is legal," Sawicki said.
"People often minimise their alcohol use until there's a climate of trust and support.
"It's hard enough with a legal substance, but with an illegal substance I think patients are often anxious and frightened to tell their doctor about how much they're using."
The medical association was not available for an interview, but admitted its stance is not the view of all doctors, or of all NZMA members.
It said it has always made that clear.
But press releases and other information shared by the NZMA does not make that statement, and the doctors RNZ talked to said not all doctors were canvassed before the association took its stance.
The NZMA said it has a robust process on policy and advocacy with members which informed its position.