Bill English is staying silent on whether he wants to be New Zealand's next Prime Minister.
But the deputy prime minister will not rule out a bid for the top job, telling reporters at Parliament this afternoon he will discuss the matter with the National caucus and his family over the next few days.
And Corrections and Police Minister Judith Collins in an interview this afternoon also did not rule out a bid to replace Key.
Collins told Newshub she had not decided whether to challenge for the top job: "I'm thinking about it. I'm not really prepared to go any further".
Collins indicated the vote to replace John Key might not necessarily happen according to the timetable previously outlined by the outgoing Prime Minister, Newstalk ZB reports.
"The caucus has only known about this for two or three hours and I think we just want to give ourselves a bit of space," he said at a crowded press conference outside the debating chamber.
Under questioning by reporters, English said he would not rule out a bid for the leadership.
"But certainly I wouldn't stand if there wasn't strong caucus support for me standing."
WATCH: Bill English speaks after John Key's shock resignation
#LIVE Would you like to see Bill English replace John Key as Prime Minister? http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=11760723
English has been in Parliament for 26 years and led National to its worst-ever election defeat in 2002. A leadership bid could raise questions about whether he is the right person to lead National into the future.
English said National had "energy and direction", no matter who its leader was. He also said he had learned "more from losing than from winning" and been given a "masterclass every day" while deputising for Key.
Key raised the idea of resigning with English in September, but did not confirm his final decision with his deputy until this morning,
"I was surprised, back in September, certainly, that he was considering it," English said. "And it's the sort of thing that because it's not happened before you don't quite believe it 'til it happens."
"I can't recall a Prime Minister who has taken this step while they are doing as well as they have ever done."
English: Key is 'one of NZ's greatest'
In a statement released earlier today, English spoke of his leader's years of dedication and outstanding service.
"John Key's intelligence, optimism and integrity as Leader of the National Party and Prime Minister of New Zealand means he will be judged by history as one of New Zealand's greatest leaders," he said.
"On behalf of the National Party, the Government and New Zealand I thank John for his years of dedicated and outstanding service to our country.
"I thank Bronagh, Stephie and Max for the sacrifice they've made to enable John to be an extremely successful and effective leader. We are deeply appreciative."
English said while Key would be leaving a gap, the party respected his decision to step down from a job "from which there is no respite".
"We wish John and his family every success with their life out of the public eye."
English said the Prime Minister had enabled the Government to ensure the country was "one of the most desirable places to live, work and raise a family in the world".
"Through good times and bad, his strong leadership has been steadfast and this is a more confident, successful and self-assured country because of his contribution. He has truly made a difference."
He said as the National caucus looked to the future, it would carefully consider how to keep the country on track to building a strong economy.
"It is a tribute to the Prime Minister's outstanding leadership that he will leave behind a united team with plenty of talent to take New Zealand forward and build on his legacy."