He has held roles at Credit Suisse, Puhoi Valley Cheese and Carter Holt Harvey and is chair of the Prime Minister's Business Advisory Council.
One fund manager said Whineray is a popular choice in the financial markets because of his track record with Mercury and because of his understanding of dairy through Puhoi.
News of the appointment comes as Fonterra' top management do the rounds with Australian institutions to talk about the company's strategy of getting back to basics after posting its second big loss in a row last month.
Mercury chair Prue Flacks said the shift was a great opportunity for Whineray, who had led Mercury to a strong position.
"Fraser has achieved a tremendous amount through his tenure, including setting the company on the next wave of its renewable energy growth path for the benefit of our customers, our communities, our owners and the country," she said in a statement.
Earlier this year Mercury announced the development of the company's first wind farm, being built at Turitea near Palmerston North.
It will be New Zealand's first major generation development since 2014.
"Fraser simplified the business through the divestment of overseas interests and the sale of its former smart metering business, Metrix; he oversaw the mothballing of the Southdown gas-fired power station in Auckland as the company turned its focus fully to renewable generation; he led a rebranding to Mercury from the two brands of Mighty River Power and Mercury Energy and he has developed with his executive team a compelling strategy for sustainable growth focusing on core pillars of the business," Flacks said.
Whineray was a unanimous choice for what was then the Mighty River Power board when it came to replacing Doug Heffernan in 2014.
He will continue with Mercury until his move to his new role at Fonterra around the end of the first quarter of 2020.
Flacks said the process to appoint a successor would begin immediately with an internal and external search.
Farmer-owned Fonterra Cooperative Group shares and units in the Fonterra Shareholders Fund have risen by over a quarter since it unveiled its new strategy at the end of September.
The units - which give investos access to Foterra's dividends - spiked up by 6.6 per cent on Monday. They last traded at $4.07 - up another seven cents - after hitting a record low of $3.15 in August.