Sergeant Matt Boyce said it was an "unprecedented event" and a tragedy and police were supporting those involved.
Police said it appeared the boat collided with something but couldn't comment on if it involved a whale. Earlier in the day Kaikoura's mayor Craig Mackle said he believed the boat had hit a whale.
Asked if it was a working theory that the boat hit a whale, police said they were speaking to a number of people to find out what had happened.
Asked if the boat was overloaded, Mackle said he couldn't answer that.
The bodies of the dead were found in the vessel.
The coastguard said it was a "tragic operation".
"Our thoughts go out to friends and family of the deceased," the coastguard said.
Tracy Phillips, Maritime NZ Principal Investigator, said the organisation had sent two investigators from Christchurch to Kaikoura.
"Any investigation activity will commence only after rescue/recovery operations have concluded. We'll be conducting a thorough investigation under the Health and Safety at Work Act and the Maritime Transport Act and we'll do everything we can to support NZ Police in their investigation."
Mackle earlier said he felt sick as the incident unfolded.
Richard Hill from Cods and Crays fish and chip shop told RNZ he had worked with the skipper on the boat before.
"He's a very safe man, there's no way he'd take any risks. So there's just got to be something sort of a freak of nature, I would have thought. A very sad day for Kaikōura really, just a shock," Hill said.
Kaikoura District councillor Lisa Bond said the incident was "incredibly heartbreaking".
The South Bay slipway was closed to the public while police responded.
A reporter on the shore at Goose Bay said at 1.45pm he could see three boats, two of which looked to be Coastguard vessels, and four helicopters in the vicinity of the overturned vessel.