Martin today told reporters one NZ First MP had at their caucus meeting announced they would be putting forward an amendment and that she went to see Little to tell him about it about 1pm on Tuesday.
"And I apologised for the fact it had never come up," she said.
That left Little facing questions about why he had told media he had not heard about the referendum at about 2pm.
It was "only a minute's difference", he said.
Little also shrugged off the debate, saying he was focused on getting his bill passed, as is.
NZ First leader Winston Peters, meanwhile, says his party has nothing to be sorry for.
He said it had long been his party's policy to seek a public mandate on conscience issues and no one should have been caught out by the move.
"We had the final … debate in our caucus after Tracey Martin said that," he said.
Earlier, he lashed out at Labour, telling Sky it was in fact NZ First that had been blindsided by the abortion reform legislation.
"It wasn't part of our coalition agreement, so why is it there? The fact that we're prepared to accept that they've put it there is a matter of good faith on our part," he said.
"If anyone got blindsided, it was us. But we didn't get upset or have a hissy fit, as some do in politics."
Little rejected he had blindsided NZ First, or thrown a "hissy fit".
"What feelings I have for Winston are feelings of respect, admiration and that's it," he said.
It's not the first time Little has been taken by surprise by NZ First.
He also believed he had their support to repeal three-strikes criminal legislation, only to find out, at the last minute, he didn't.
New Zealand First is also insisting on a referendum on David Seymour's End of Life Choice bill, which would legalise assisted dying for terminally ill adults.
A referendum is already being held at next year's election on recreational marijuana legalisation.
