He said he fished the area regularly and on Sunday there had been about a 1m swell.
Wearing lifejackets was mandatory on his boat, he said.
Once in the water they made a decision to stay with the boat. While the boat was drifting towards an island they decided not to send a swimmer to shore.
"We made a blanket decision to stay with the boat."
Someone on shore spotted the overturned vessel and went out in a boat and plucked them from the water.
Two of the men were treated for mild hypothermia by St John staff on the beach.
Police Search and Rescue officer Constable Sue Grocott said there was not much you could do about a freak wave but the lifejackets probably saved them.
"The fact they had their life jackets on was brilliant."
Ms Grocott reiterated the boating safety message to always wear lifejackets, have some form of communication other than a cellphone, have warm clothes and have outboard motors checked before heading out on the ocean this summer.
The Whangaroa Coastguard team were on their way but the four men were taken to shore before they arrived.
However they righted the boat and towed it to a sheltered area where they pumped out the water and then took the boat to the Whangaroa Harbour marina.
Earlier this month a 58-year-old man drowned off Matapouri Beach.
He was with a 27-year-old man on a 14ft catamaran launched from Matapouri beach which then capsized.
The death took the region's drowning toll to nine this year.
Fourteen people drowned in the region in 2011, up from nine deaths in 2010 but below the 2009 total of 16.