"We've known it's coming, but when you're coming along on your scooter on the way in past the open expanses and you're getting blown around, you know they're going to be in for a bit on the race course," he said. "We're just a bit mindful of making sure they come back in one piece."
Hours later, their collective fears were realised, when the New Zealand catamaran violently pitch-poled off the startline in their second match-up of the day against British team Ben Ainslie Racing.
It looked like it was happening in slow motion for the viewers, but it wasn't for those on board. Three of the team's cyclors, Blair Tuke, Andy Maloney and Josh Junior, were sent flying at high speed into the water.
Helmsman Peter Burling, who along with Ashby and cyclor Simon van Velthooven, was able to cling on in the cockpit, later spoke at the relief of looking down into the whipped up waves on the Great Sound and seeing his teammates were all okay.
"I was very thankful to be sitting up the top at the back and see all their heads in the water and know they are all safe," he said.
Back on land, shocked family and friends of the team watched on in horror as the moment unfolded. Mel Ashby, wife of the Team NZ skipper, said she initially struggled to process what she was seeing. While she got her head around that, she was trying to reassure the children in the group it would all be okay.
"It happened so quickly. Then when the kids recognised [what was happening] ... it was scary for them. There are some dads out there, husbands, sons, friends," she said. "And the kids love that boat as well."
As that boat limped back to shore, barely visible through the squalls that whisked through the Great Sound, the shore team gathered at the dock, standing silent vigil as they willed the cat to come back in a reparable state.
The shore team had already been placed under incredible stress yesterday, having pulled off a late swap-out of the wingsail after Team NZ sustained damage soon after getting out on to the course. The Kiwi team were able to somehow get back out on the water and into the startbox in time for their first race of the day against the Brits. Even more remarkable still, the crew were able to regain their focus and claim a gutsy victory over BAR in a heart-in-mouth high-speed duel.
The racing had made for uncomfortable viewing all afternoon, as the boats teetered on the edge of control in the blustery conditions.
In the other semifinal match-up between Artemis and Team Japan, there were pieces of carbon fibre flying off the yachts as they bucked and reared their way around the nine-lap course.
"It is like skiing on ice," a beaming Ainslie later explained, describing the racing as the most "exhilarating and exciting" of his career.
His enthusiasm could also have been on account of a place in the challenger final looking a lot more achievable than it did at the start of the day.
Burling, meanwhile, whose background is in high performance skiffs, could find little about the day to be exhilarated about. He looked wrung out when he fronted the media conference less than an hour after returning to the dock - the events on the race course still weighing heavily on him. But even after yesterday's setback, Burling was not willing to give up on Team NZ's 2017 campaign.
"We'll have to dig pretty deep to get out there again, but we feel like we can do it," he said. "As a group of New Zealanders, we are incredibly resilient."
Results
• SoftBank Team Japan bt Artemis Racing by withdrawal (Japan lead 2-1)
• Emirates Team New Zealand bt Land Rover BAR by 2m 10s (NZ lead 3-0)
• SoftBank Team Japan bt Artemis Racing by 27s (Japan lead 3-1)
• Land Rover BAR bt Emirates Team New Zealand by walkover (NZ lead 3-1)
