Shortly after 10am on Saturday, an 8.5m boat, with 11 people on board, capsized in "perfect, flat" conditions after reportedly hitting a whale.
Police could not confirm it was a whale that caused the boat to flip, saying the vessel had collided with "something" but they were speaking to a number of people to find out what had happened.
However, earlier in the day Kaikōura's mayor Craig Mackle said he believed the boat had hit a whale.
Six people were reported alive, with one person receiving minor injuries.
They were taken to hospital before returning to their accommodation, police told media at a press conference shortly after 5pm yesterday.
The police dive squad recovered the five who had been found dead in the vessel, Sergeant Matt Boyce said.
"We are in the process of identifying those people and our next job is to notify their next of kin."
He said it was an "unprecedented event" which involved a significant response from emergency services and members of the public.
"This is a tragic event and police are providing support to those involved at this very difficult time," Boyce said.
The passengers were believed to be part of a bird enthusiast group who hired a charter boat to visit areas of interest.
"I believe they had come from all parts of the country," Boyce said.
A local Kaikōura resident who assisted at the scene said they knew one of the victims - who was connected to their son's school.
She went to the shore to help with the search.
"It wasn't much, but more than I was adding being at home.
It was difficult to see clearly along the coast due to the large number of seals sprawled along the rocks, she said.
"This will be a horrific time for the families and will be felt throughout our close community tonight and years to come," she said.
"We didn't even lose five people in our 2016 earthquake."
Police could not say who the charter was operated by. The skipper went to hospital but police do not believe he was injured.
The pilot of the first rescue helicopter at the scene said he saw no one in the water when he arrived.
"It's a terrible thing - it's something you don't want to see with people in the water," Kaikōura pilot Daniel Stevenson said.
Just before he arrived, a pleasure craft got to the scene and grabbed the first group of people that were outside the boat and took them back to Kaikoura.
According to Stevenson, five people were rescued, and one person was left, sitting on the upturned hull of the boat.
The coastguard arrived shortly afterwards. They jumped into the water but couldn't get under the boat and requested a diver.
Stevenson flew back to Kaikōura, picked up a diver and hovered beside the upturned vessel.
"They jumped into the water out of the helicopter to see if there was anyone trapped inside the vessel," Stevenson said.
The mayor said it was a tragic event that affected many lives, not least of all family and loved ones.
"Our heartfelt sorrow and condolences to you all."
Mackle said he wanted to thank everyone involved in the rescue and recovery.
"Being able to bring everyone home is the best result in such a terrible circumstance."
Whales were resident and had been seen in the area, Mackle said.
Sea conditions at the time were "perfect, flat", he said.
Asked if the boat was overloaded, Mackle said he couldn't answer that.
The coastguard echoed that it was a "tragic operation".
"Our thoughts go out to friends and family of the deceased," the coastguard said.
Richard Hill from Cods and Crays fish and chip shop told RNZ he had worked with the skipper on the boat before.
"He's a very safe man, there's no way he'd take any risks.
''So there's just got to be something, sort of a freak of nature, I would have thought. A very sad day for Kaikōura really, just a shock," Hill said.
Tracy Phillips, Maritime NZ Principal Investigator, said the organisation had sent two investigators from Christchurch to Kaikoura.
"Any investigation activity will commence only after rescue/recovery operations have concluded. We'll be conducting a thorough investigation under the Health and Safety at Work Act and the Maritime Transport Act and we'll do everything we can to support NZ police in their investigation."
Kaikoura District councillor Lisa Bond said the incident was "incredibly heartbreaking".
The South Bay slipway was closed to the public while police responded.
A reporter on the shore at Goose Bay said at 1.45pm he could see three boats, two of which looked to be Coastguard vessels, and four helicopters in the vicinity of the overturned vessel.
Three rescue helicopters were sent to the scene, one from Wellington and two Westpac Rescue Helicopters from Christchurch.
An image on social media showed a number of people on top of the upturned boat, awaiting rescue.
