Tiaan Falcon converted from a handy position to secure the home side's third bonus point win of the season and put them seven points clear of Otago in the Championship division with the Steamers currently third of the seven teams.
But it was a three minute period early in the second half which, in the words of McMillan, "turned the game on a dime".
Bay of Plenty had taken a 13-8 lead into half time and playing with the wind in the second half, hoped to extend that advantage.
But one test All Black Brad Weber scored two quick tries, both of them converted by Falcon, and Hawke's Bay had turned a five point deficit into a nine point lead.
The legality of both tries could well be questioned.
For the first, Hawke's Bay blindside flanker Marino Mikaele Tu'u picked the ball out of a back-pedalling scrum to feed Weber. Scrums are places where only feet are supposed to play the ball.
Weber's second try came after he took a pass from winger Jonah Lowe at least a metre in front of where Lowe passed it from.
What's more Weber was tackled, although not held, short of the line. Instead of regaining his feet, as one is supposed to, he crawled across to force the ball.
McMillan wasn't prepared to comment further on those incidents, although he did suggest that "there are quite a few times in the coaches' box when you throw your arms in the air".
Instead, he's ruing the Steamers' missed chances.
"We just weren't ruthless enough to turn opportunities into points.
"We have to respect the ball a lot more and we need to be a lot smarter with the ball we have."
The Steamers ran more metres in the match, made more clean breaks and completed more passes and offloads than Hawkes Bay. Sadly for them, only one statistic counts – the final score.
While Weber scored two decisive tries for Hawke's Bay, the Steamers had their own try scoring hero in the No 9 jersey.
Richard Judd darted across from the back of a scrum in the first half to help his team to their 13-8 lead.
Then he sparked the Steamers comeback at the three quarter mark of the match with a brilliant solo effort.
Picking up the ball from a ruck, he ran a full 30m to score but was hit heavily in the act of getting over the line and left the field almost immediately.
The Steamers next match is against North Harbour at Albany on Saturday afternoon.
Scorers:
Hawke's Bay Magpies 29 (Brad Weber 2 Jonah Lowe, Joe Apikotoa tries; Tiaan Falcon 3 cons, pen), Bay of Plenty Steamers 28 (Richard Judd 2, Mike Delaney tries; Delaney 3 pens, 2 cons). HT: Steamers 13-8.
