Fonterra last month named long-serving director John Wilson as van der Heyden's replacement.
Wilson, who has been a director of the cooperative since 2003, will take up the new role when van der Heyden steps down at the annual meeting in December.
Armer was seen by many farmers as the logical next choice of chairman.
Wilson, on the other hand, was understood to have been van der Heyden's preferred choice as a successor.
Van der Heyden said Armer had made "a significant contribution'' to the progress of Fonterra. "While we are disappointed that Colin has chosen to resign we respect his decision,'' van der Heyden said in a statement.
On being included in last week's NBR Rich List, Armer told the Bay of Plenty Times that he and his wife made their money by working hard and investing in their business Dairy Holdings - New Zealand's largest dairy farming conglomerate.
"It's been hard work between both of us,'' Armer told the paper.
He is one of four directors of Dairy Holdings, of which he and Mrs Armer are majority shareholders.
In February, the couple were instrumental in ending the sale process of Dairy Holdings, which was then partly owned by failed finance company, South Canterbury Finance.
The Armers, and a few other investors, bought South Canterbury's $54.5m stake, giving them full control of 58 South Island farms, covering 18,000 hectares and 43,400 milking cows.
Last month, dairy farm company Armer Farms (NI) Ltd was fined $72,200 for the unlawful discharge of dairy effluent near Maketu, the Bay of Plenty Times said.
The couple were directors of the company but were not charged because they did not have day-to-day responsibility over the farm.
Armer was elected to the Fonterra board in 2006 and was chairman of the supplier relations committee, the international farming ventures group, and the milk payments working group.
He also sat on the audit, finance and risk committee, the appointments, remuneration and development committee and the external relations committee.
Farmer shareholders will vote on a new director at December's annual meeting.