"I hear what the member is suggesting, and I will discuss that with colleagues," Little said.
Labour and National closed ranks on Tuesday, reiterating their promise to swiftly pass a change to New Zealand's counter terrorism laws in light of last Friday's attack.
The Greens and Act took a different tune, condemning the terrorist attack, but urging Parliament to take its time before legislating a response.
MPs debated a Ministerial Statement on Tuesday. These are special, free flowing, debates that give MPs the opportunity to ask Ministers probing questions on areas of Government policy.
Little, who has responsibility for terror laws as minister in charge of the security agencies and the Government's response to the attack on the Christchurch Mosques, said it had been known for some time there were holes in our terror laws.
Little said that as early as November 2007 "the then Solicitor General found the Terrorism Suppression Act 2002 to be unworkable".
The problem, Little said, was that it "was found not to contain a criminal offence of planning or preparation for a terrorist attack".
He said work did not begin on plugging that hole until August 2018, and the results of that work are currently before a Parliamentary select committee.
He defended the time taken between August 2018 and now to get the law into shape.
"It was a very significant piece of work because of the interplay between security, individual freedoms and human rights," Little said.
Collins urged the Government to consider whether anything further could have been done to deport the terrorist, saying the event raised "unique issues around refugee status and immigration law in New Zealand."
"We have international obligations to ensure the rights of refugees are upheld, and we have moral obligations too.
"However, it is my strongly held opinion that the Government of New Zealand must prioritise keeping New Zealanders safe above all else.
"If a refugee or an immigrant commits an act of terror or goes overseas to take part in extremist activities, we should rip up their New Zealand passport," Collins said.
Green co-leader James Shaw said Parliament should find the "best long-term response" to the attack.
"Unless the Government knows of another clear and present danger that is likely to crystallise in the coming weeks, the use of urgency to rush through legislation should be opposed on the grounds that it is more important to get right than it is to be seen to be doing something," Shaw said.
"Rushed legislation in response to events runs the risk of increasing harm to innocent people, particularly Māori, migrants and refugees," he said.
Act leader David Seymour said the "the last thing we should do is abandon the very values that make this country great - in particular our system of deliberative, consultative Parliamentary democracy".
"We should not replace rushed and bad laws with more rushed and bad lawmaking," Seymour said.
