Organisers had aimed for 1000 people to sign up. Once that milestone was achieved, they increased the goal to 2000.
As of 7pm, almost 2500 people had signed the petition.
"Imagine how powerful and healing this moment could have been - if these brave women had been allowed to finish their sentences, taking a stand for all survivors of sexual violence, and the Prime Minister had simply stood and said, 'I'm sorry'," the petition states.
"Instead many of the MPs were prevented by the Speaker from completing their statements, several were thrown out of the Chamber, and others left in solidarity. It was, as news outlets have observed, a disgraceful day in New Zealand's House of Parliament."
Organisers called for the public to stand up and "tell John" to apologise and show support for real rape survivors.
"We need to tell John Key that it's not okay to throw around accusations of 'backing rapists' as a cynical political tactic when for too many women, and men, in New Zealand rape is very real, and very painful," the petition states.
Mr Key said Labour were "backing rapists" on Christmas Island, in a dramatic outburst in Parliament on Tuesday.
The Prime Minister remained defiant about the accusation on Wednesday, sparking outrage among MPs.
Several of them, including Greens' Metiria Turei, Jan Logie, Catherine Delahunty, and Labour's Poto Williams revealed that they were victims of assault and had taken personal offence at the Prime Minister's accusation.
Despite a warning from the Speaker, MPs continued to stand to urge him to take action.
Green MP Marama Davidson was then thrown out of the House, followed by Poto Williams.
They were followed by at least eight others who chose to leave the debating chamber.
Labour MP Clare Curran said the Speaker's decision to evict some of the women who had raised their own experiences sends a bad message.
"The Speaker today threw out women from the House of Representatives for standing up and saying they were the victims of sexual violence. What message does that send to young women?"
The petition is at: http://www.actionstation.org.nz/saysorryjohn