Whitianga Police co-ordinated the rescue which included the Pauanui Coastguard and Fire Rescue, and the Westpac Rescue Helicopter.
Recreational boaties and charter services also joined the search for the man. Diver Dirk Immenga said a doctor was dropped from the helicopter on to the Coastguard boat to perform CPR when the man was found.
The boat raced back to shore shortly before 2.30pm where resuscitation attempts continued.
Three Auckland schoolgirls who saw the rescue on Hahei Beach said it was sobering. Twins Grace and Freya Burnett and friend Kate Burtson were kayaking when the Coastguard boat returned to the beach.
"He was being given CPR on the Coastguard boat. They stopped after 20 minutes and pulled a white sheet over him," Freya said.
"It was really sad. It was awful. You sort of expected that he might be saved." Beachgoers rushed to aid the rescuers but their efforts were in vain.
Mr Cochrane said South Sunk Rock was one of the more dangerous diving spots in the area but was popular because there was a 10m drop-off behind the partly submerged rock.
The man was diving from a private boat, thought to be a cabin cruiser, with his brother. Police could not confirm unofficial reports there were children on the boat.
It's unclear if the man was married or had children, or if he lived in Whitianga.
Dive Zone Whitianga manager Darrell Bird, who helped co-ordinate the rescue, said he knew the man's family, including his brother.
He said the dead man was known to those involved in the rescue operation.
"Everyone was out there. Our thoughts are with the families now and we just want to let them have some time."
