Mr Scott said they kept asking what happened, was there anyone else and were they under the boat, but the man was in shock and couldn't speak English very well.
Eventually, he was able to tell them he had two friends with him. When asked where they were, he said they were "gone but that they were both together with lifejackets".
Mr Scott called the Coastguard, which sent its Air Patrol plane and the NZ Steel rescue boat to try to find them.
While in constant communication with the Coastguard, Mr Mankelow, the skipper, marked the spot on his GPS and zigzagged the boat back to where they thought the others might be, but couldn't see any sign of them.
The Air Patrol plane spotted the other two men clinging to a chilly bin in the breakers by the bar.
Once the men were pulled safely into another fishing boat, the man in Mr Mankelow's boat waved at his friends, prayed and cried with relief.
The men had been in the water for almost two-and-a-half hours.
"They were pretty bloody lucky. I don't think the first guy realised how bad it actually was," Mr Scott said.
"And it was lucky we went with our gut and went to check out what it was in the water because we were probably the last lot of boats coming in."
Two of the men were hypothermic and the third had chest pains and sore arms and legs.
All three were transferred to the NZ Steel rescue boat and taken to shore to waiting ambulances.
A St John spokesman said the men were taken to Waitakere Hospital, one with moderate injuries and two with minor injuries.
Mr Mankelow said the men's boat seemed to have proper communication equipment but they probably didn't know that it was protocol to call the Coastguard before and after making a crossing. He said it was fortunate they had spotted the men in the water.
Mr Mankelow's boat tagged a marlin earlier in the day, something, like the rescue, he puts down to luck and timing.
"Being part of a rescue as a member of the public is just the same as hooking up a marlin - you've just got to be lucky. It's all about timing. I see it no different to fishing."
Coastguard spokeswoman Georgie Smith said if the men had sent through a bar-crossing report, they would have been found much sooner.
A bar-crossing report lets the Coastguard know when a boat is about to pass through the bar. Boats then check in again once they have successfully crossed.
Ms Smith did commend the men for wearing lifejackets.
Staying safe
Coastguard's advice on how to cross a bar safely
• Seek local advice on the bar and channels
• Check the weather, tide and bar conditions
• Ensure your boat has adequate stability
• Check your boat and make sure hatches and equipment are adequately secured
• Make sure that everybody is wearing a lifejacket and is awake
• Ensure that you are carrying two forms of communication that work even when wet
• Make a bar-crossing report with Coastguard radio