Bridges said that this type of legislation was "so fundamental to society that it can't be rushed or fast-tracked".
"Freedom of speech is so incredibly important to us as a society."
He added that there are many things he does not agree with – such as the comments from the Hamilton city councilor James Casson, who took to social media to say each time Prime Minister speaks of hate or terror, the accused Christchurch gunman "wins".
But Bridges said New Zealand was a free society and people had the right to the freedom of speech.
He said videos that incite violence should be banned as they were "clearly across the line".
"But you know what? I also know that for centuries people have fought for freedoms. In New Zealand, at the moment we believe strongly… in freedom of expression."
"I think actually that's what Andrew Little believes as well, and that's why I have no criticism of him having a review and going through this, albeit you have to be careful – you can't do away with the right to freedom of speech or change it at all lightly.
"Where that line between free speech into that hateful and incitement of violence and the like is, is not easy."
Bridges said it is important that the Government "treads lightly on this".
Act Party Leader David Seymour is also advising caution on any changes to rules that might impact free speech.
"When the Government makes it its role to start working out which opinions are right and which are wrong and which ones should be punished, that's when you get into real difficulty," he told RNZ.
He said a new hate crime law would exacerbate divisions and fail to stamp out prejudice in New Zealand.
