The Conservatives achieved 3.97 per cent of the vote in last year's election - more than 95,000 votes, but short of the 5 per cent needed to enter Parliament.
"You just dont know how all this stuff is going to play out. He has been the big funder of the party. We are two and a bit years from the election. It's a long way to go," Mr Key Mr Key told the Paul Henry show.
"Those Conservative voters...if they [the party] were not so strong a voice - they have well come back to us."
Meanwhile, Mr Craig says he has received hundreds of supportive messages from party members and will listen to them and not the board members who oppose him.
Mr Craig will this morning consult lawyers and send a formal warning to board member John Stringer over comments he has made about the party founder's conduct.
Mr Stringer has been the most outspoken critic on the Conservatives' 11-person board, but other key backers are withdrawing support from Mr Craig, including big financial donor Laurence Day.
Mr Stringer has said there has been widespread concern from party members about the "awkwardness" of the relationship between Mr Craig and his former press secretary, Rachel MacGregor.
He has thumbed his nose at Mr Craig's legal threat and said support among board members for the return of their leader had "vapourised" over the weekend.
"I have been talking to the board. Colin has zero support. He won't be [reinstated]," Mr Stringer told the Herald.
Mr Craig's brother, Andrew Craig, is a board member. Mr Stringer said he could not speak for him, but "Andrew is very supportive of my position".
"I know that he along with other board members are very unhappy with Colin's behaviour."
Andrew Craig would not be drawn when asked if he would like to see his brother return as leader.
Another board member, Laurence Day, said he was not aware of any board member who wanted to see Mr Craig recalled as leader. "I don't see, at the board level, any great enthusiasm for him to return.
"You have to respect the guy for what he's done, but ... I can't speculate on the veracity of what's going around."
Mr Day and his wife, Katrina, gave $675,000 to the Conservatives' election campaign. He said he would still support the party.
Mr Craig said he had written to Mr Stringer asking him to retract his comments, and would take advice on a more formal approach today.
"I think once he receives that it might be wise for him to pause ... and then maybe reconsider his position."
Asked if he would seek the leadership if there was support from party members but not from the board, Mr Craig said he would, provided "whatever allegations are out there are cleared up".
