"Every dollar spent should come with the questions: 'Does this help the athlete win? Will it help improve results?' Those are the areas the money should be going to."
Drysdale, Aleh and Adams want to sit down with country's sports chiefs and start a dialogue about the way forward for sports funding.
Top of the agenda is a structure that saw 85 employees of Sport New Zealand and High Performance Sport New Zealand, on salaries of more than $100,000 in 2015.
In simple terms Sport NZ is in charge of sports participation, including recreation and grassroots, and controls government funding while HPSNZ is charged with helping our sportsmen, women and teams win on the world stage.
Forty-one of those six-figure salaried employees work at HPSNZ.
HPSNZ chief executive Alex Baumann earned $420-430,000 in 2015, a figure that puts him just below the Prime Minister's salary. Peter Miskimmin earned between $380-390,000 as chief executive of Sport New Zealand.
Athletes receive performance enhancement grants (PEGs), which are taxed, ranging from $60,000 for a gold medallist, to $55,000 for a medallist to $25,000 if you're top 12 in the world.
Athletes also have access to coaches, sports science, sports psychology, medicine, nutrition and physiotherapy.
Some high-profile athletes like Drysdale are able to secure endorsement deals to top up their income, but most prospective Olympians are faced with the reality of working while training.
The disparity between what our best athletes receive from the taxpayer and our chief administrators receive, is stark.
Aleh, who along with Polly Powrie won a remarkable silver in Rio despite two race disqualifications to back up gold in London, said the key word for her was "clarity".
"We [the athletes] need to see how the money is spent. That clarity is not pushed from the top. If it was more open and transparent perhaps we'd come to the conclusion that they're doing a good job.
"We're not doing this for money. If that was why we wanted to compete, we'd be in it for the wrong reason. We all want the same thing. They want medals, we want a medal. It's how we can better work together better."
Drysdale said the last time sports funding was reviewed he "was waiting for a call" that never came. He says it is vital that athletes start to have a say in how the money is spent.
Baumann described the issue of funding as a delicate "balancing act".
Of the $56m spent each year, $34.5m went to national sporting organisations to run programmes, while $7.3m went directly to athletes in the form of PEGs. A further $4.25m was split between 394 athletes on Prime Minister's Scholarships.
"There should be transparency. I'm happy to explain where the money goes and that is an issue of better communication," Baumann acknowledged.
Both Aleh and Drysdale said they were appreciative of what they received now given they started coming through the system more than a decade ago when athlete grants were either non-existent or far less than they are today.
Adams, whose glittering career has seen her win two golds and get within centimetres of a third, has had the luxury of being able to carve out a tailored programme to suit her specific needs, largely free of interference.
She told the Weekend Herald in a statement from Rarotonga that hers was a model that could be applied to more athletes across a range of sports.
"I have a set-up through Athletics New Zealand that works well for me but it wasn't always like that. In 2008 I identified that we had to make major changes.
"High Performance Sport could look at some of the learnings we developed from the changes we made in 2008 and this model could possibly be set up for other athletes that may work better for them."
The athletes point to a "bureaucracy" that sucked up a lot of money that could be better spent elsewhere.
"Obviously I'd like to see more money directed to athletes and coaches, not systems, but it's not a bad thing that support staff get paid well. You want the best people and the best people tend to cost more," Aleh said.
Said Baumann: "We could give everything to the athletes but we wouldn't get the results like we did in Rio.
"We have a pretty progressive system," he said, referencing the fact there is no means testing on athletes as there is in Australia.
Several athletes in addition to the medallists highlighted here have instructed their representatives and the Athletes' Federation to push hard for change, believing New Zealand's taxpayer-funded high-performance spend is going to the wrong people and the wrong places.
"There's a big difference between being on a salary and on a PEG," said Rob Nichol, chairman of the Athletes' Federation.
"It is a fundamentally different proposition because your income can drop overnight.
"You can't go to a bank and get a mortgage based on PEGs," he said.
Baumann said PEGs were never designed to act as a salary, but were a critical part of the high-performance system as they gave selected athletes more flexibility to be able to train fulltime.
"They are not employees of High Performance Sport New Zealand," he said. "PEGs are not salaries, but they're a fundamental tool."
To the criticism that an athlete can see their PEGs drop dramatically due to one bad performance: "You can't get away from performance accountability," Baumann said.
"Sometimes it seems harsh but I'm a strong believer in that."