"Wind was going to be a factor because we knew we didn't play well enough in the first half. We wanted to keep it closer than it was," said the veteran, disclosing the visitors were hoping to use that stiff breeze from the city end along the field to their advantage.
Bay United striker Sam Mason-Smith scored in just the second minute after a defensive blunder although Ifill had made ground from across the left flank in a failed attempt to avert disaster.
However, Tasman United's Tinashe Marowa levelled, 1-1, in the 11th minute when midfielder Ryan Stewart intercepted a misdirected defensive pass, dribbled the ball past a couple of defenders to push it to the talented striker from Auckland to slip it under Bay goalkeeper Joshua Hill.
Six minutes later the hosts went up 2-1 again from Mason-Smith who scored an opportunistic goal when he dutifully followed up a drive from centre-mid Ross Willox from 25m out but Tasman goalkeeper Thomas Fawdry fumbled in front of the goalmouth.
Bay United extended that lead to 3-1 in the 19th minute, when former import player Scott Gannon, an Aussie, tripped Argentine striker Facundo Barbero in the 18m box.
Referee Anne-Marie Keighley instantly pointed to the spot but Fawdry thwarted Bay top goal scorer Saul Halpin's penalty kick although the English striker pounced on the deflection to slot it into the net.
Keighley then ran to her assistant, Ashton Davenport, who had his flag up.
After some deliberation she awarded the penalty-kick goal to Halpin amid suggestions the goalkeeper had moved over the line before the kick was taken.
It was the turn of Tasman coach Richard Anderson to start howling on the sideline at fourth official Anthony Riley.
Leftback Fergus Neil, in place of import Kohei Matsumoto who has jetted back to Japan because of an illness in the family, almost scored a goal on the stroke of halftime but the shot from 23m out hit the crossbar and jagged back into play over the line.
But it was Tasman who made their intentions clear when they scored a superb goal to make it 3-2 in the 52nd minute.
Rightback Cameron Lindsay made a diagonal cross from the grandstand side of the park, about 45m out, to inside the 18m box on the left flank where Marowa deftly volleyed it with his right foot inside the far right upright despite a desperate dive from Hill.
Tasman dominated and threw the kitchen sink at Bay United but Milne marshalled the defence to deny them any points.
"Outwardly we didn't do enough with too many mistakes and to go behind in a little more than 90 seconds isn't good enough," said Ifill, admonishing himself for making a good tackle but gave the ball away that resulted in the first goal.
He felt Tasman were the better side in the second half in creating numerous chances, which was the "ridiculous" theme of their sorry saga this summer.
Mindful he would get into trouble if he said too much about the officiating, Ifill said: "It had nothing to do with whether the referee was a male or female but she was just out of her depth today.
"Some of the decisions were just baffling," he said, adding her assistants, including Bay's Gareth Sheehan, were terrible.
"He's [Davenport] pointed one way and it's clearly the other. Just make a decision and let's get on with it. And then you're trying to talk to me and I don't want to talk to you," he said.
Referring to the penalty-kick commotion, Ifill said he queried why play was halted after the whistle went but the officials were clueless.
"He didn't know what was going on. She didn't know what was going on. She doesn't know what the rules are so how do you work with that?
"I know I'm going to get into trouble for this but I asked her why did you blow the whistle [before the ball went into the net] and [she said] 'I don't wanna tell ya', so that was the end of the conversation.
"I just wanted to know what her problem was."
Ifill said at halftime they informed him what the problem was.
"You could have told me that eight minutes ago and it would have stopped everyone going around trying to kick everybody."
Ifill said at halftime the explanation was the keeper had encroached and the officials were deliberating whether the whistle had gone before or after the ball was put in the net.
"The whistle's blown, it's blown. It's a goal but, hey, that's the least of our worries," he lamented as Tasman sit on the last rung of the premiership table with just one win.
It brought a smile to Ifill's face talking about the brilliance of young Marowa, an Aucklander who came to their trails this season with just two bags of rice tucked under his arm to stay in the flat.
"He's absolutely brilliant for us and has been our best player this season," he said, revealing there's interest from A-Leaguers Wellington Phoenix and he would rubber stamp his credentials.
"The second goal from him was just outstanding, no ridiculous. He's so composed he's just tucked it in the corner," he said of the striker who has earned the nickname of "Uncle Ben".
He said Tasman played in Auckland a fortnight ago where the "world-class pace" Marowa caught the eye of the coaching stable but Ifill pointed out he was one of their youth talents.
"We picked up one of their good ones there."