Restieaux says the orcas arrived just two minutes after he and his friends had been swimming in the same spot.
"We didn't know what it was to start. We just saw a couple of fins in the air," Restieaux said.
General manager of the National Aquarium of New Zealand Rachel Haydon said the orcas were likely coming close to shore to feed.
"They're very intelligent animals and incredible hunters. When they come close to shore it's because they have likely been herding stingray in to catch them," Haydon said.
Under the Marine Mammals Protection Regulations of 1992 no one can swim within 100 metres of a whale and a vessel cannot be within 50 metres for its own safety as well as that of the whale.
Haydon recommends people swimming in the water steer clear of orcas, do not circle them, do not obstruct their path, do not cut through a group of multiple orcas, and avoid sudden movements or noises.
"They are not naturally aggressive but they are an animal in the wild and people need to be mindful," said Haydon.
Restieaux's video follows a series of sightings of orcas swimming close to shore at Ahuriri beach a few days before.
There are fewer than 200 orcas in New Zealand waters.