The great white shark appeared to be a juvenile and was approximately 2.5 metres long.
The shark came as a shock for Gray, who travels to Matatā often but had never seen a great white there before.
"I definitely didn't expect it, I expected a bronze whaler if anything.
"Me and my mate were out the day before in kayaks in the same spot," he said.
"Never seen one before around here, only baby hammerheads or bronze whalers. It was an exciting experience."
The ordeal attracted a small crowd of excited onlookers, many of whom were locals.
"The people were all pretty freaked out and amazed at the same time as they'd been swimming there not long ago. They were locals and said they have never seen a great white around before that day," Gray said.
The shark sighting comes after a warning from the Department of Conservation to remain vigilant after an increase in shark sightings around the Tauranga area recently.
Reports of great whites have increased since May 2020 and although an estimate of the number of sharks in the area cannot be confirmed, photographic ID so far has identified six individual great whites, DoC states.
"The large number of sightings are not only in the Bowentown area of the Tauranga Harbour but also along the coastline," marine biologist Melissa Kellett said.
Kellet has been studying sharks in Tauranga for five years.
DoC marine expert Clinton Duffy reminds swimmers and beachgoers to be aware of what is happening around them.
"If you are visiting the ocean you need to be a little bit vigilant of what's happening around you and swim where there are surf lifesaving patrols, and don't swim or dive alone," he said.
"If you are heading out on the water exercise caution and avoid swimming in the main channels where there are a lot of birds diving or belaying from kayaks and jet skis when fishing."
Great white sharks are protected under the Wildlife Act 1953, which means it is illegal to hunt, harm or kill the sharks. It is not illegal to accidentally catch a great white shark, but it must be released without causing it further harm, and it is a legal requirement to report the capture.