The Kermadec Islands have had a marine reserve around them since 1990 but this decision will extend it from 12 nautical miles to the 200 nautical miles of New Zealand's exclusive economic zone (EEZ).
Mr Key said the sanctuary would mean no fishing, commercial or recreational, and no prospecting or mining in the area.
As well as marine life, the sanctuary would protect the ocean floor, Mr Key said.
"The Kermadecs contain the world's largest underwater volcanic arc, and the second deepest ocean trench at 10km deep - deeper than Mt Everest is tall."
A Cabinet paper on the establishment of the Kermadec Ocean Sanctuary says that while the protected area will comprise 15 per cent of New Zealand's EEZ, there is little viable commercial fishing in the area so no compensation should be paid.
"But the mining resources are potentially very, very large there, because there are unknown quantities of silver and other resources there," Mr Key said.
"On the other side of the coin, we are also trying to protect what is, from a geological perspective, a very important and significant part of the world."
Mr Key said he thought in decades to come New Zealanders would judge the decision to set-up the sanctuary as the right one, just as today's generations viewed the establishment of national parks such as Fiordland and Tongariro.
The sanctuary will cover 620,000 square kilometres, an area twice the size of New Zealand.
Mr Key made the announcement during the "leaders' week" at the UN General Assembly in New York.
"New Zealanders value our coasts and oceans, which are an important part of our culture, economy and environment, and we are committed to managing them sustainably.
"Creating protected areas will support not only our own fisheries but those of our Pacific neighbours."
Groups that have campaigned for the sanctuary include the Pew Charitable Trust, WWF New Zealand, the Royal NZ Forest and Bird Society, Greenpeace and Ngati Kuri.
Minister for the Environment Nick Smith said the sanctuary would require legislation and the aim was to have it in place by October 1 next year.
He said the decision was taken now to give certainty to a Canadian company, Nautilus Minerals, which had applied for a prospecting permit covering an area that was partly within the sanctuary.
Officials would work with the company to revise its application.
Dr Smith said New Zealand had committed to having 10 per cent of coastal and marine areas conserved by 2020, and already had 9.7 per cent of its territorial sea (within 12 nautical miles) protected.
Ten new marine reserves were created last year: around the subantarctic islands, Akaroa, Kaikoura and the West Coast, making a total of 44 marine reserves.
Forest and Bird advocacy manager Kevin Hackwell said the announcement was "a really significant move" which would give the Kermadecs a much higher profile.
The Green Party said it was delighted with the announcement, which lifted marine reserves in New Zealand's territorial sea and Exclusive Economic Zone from 0.4 per cent to 15 per cent.
However, the party's environment spokeswoman Eugenie Sage said there was more work to do, including protecting the Maui's Dolphins, improving water quality and lifting New Zealand's "embarrassingly weak" proposal to limit greenhouse gas emissions.
"That said, good start National Government. We certainly want to see Nick Smith's discussion document on marine protection include provision for other large scale no-take marine reserves in the Exclusive Economic Zone."
WWF-New Zealand's chief executive Chris Howe said the creation of the massive ocean sanctuary put New Zealand "back at the forefront of marine protection on the global stage".
"[It] will allow some of the ocean's most exploited and threatened species to not only recover but flourish."
Reserve v sanctuary
• What's the difference between the reserve and the sanctuary?
• The rules are stricter in a reserve.
• There is no fishing or mining in both but discharges from vessels will be allowed in a sanctuary and submarine cables are allowed, whereas they are not in reserves.
• Ships are able to travel easily through the EEZ.
