The controversial decision by Massey University has added fuel to the free speech debate in New Zealand.
In an earlier statement, the university referred to Brash as the Hobson's Pledge founder and supporter of controversial Canadian speakers Lauren Southern and Stefan Molyneux.
Brash is also a member of the Free Speech Coalition which was formed after a decision was made by Auckland Council to ban the speakers from its venues.
The coalition argued the move on the part of the council was a breach of the principles of free speech.
Brash told Larry Williams on Newstalk ZB he was defending the right to freedom of speech, not the opinions that Southern and Molyneux share.
"The irony is, I to speak about my time as National party leader between 2003 and 2006, I wasn't talking about current events at all.
"In my speech notes, I make no reference to the Canadians or to the Hobson speech or anything else," he said.
Brash believed Thomas kicked him from the event plainly because she didn't share the same views as him.
Prime Minister Jacinda Arden weighed in on the Brash and Massey University conflict, stating that it was an "overreaction on the part of the university".
"There'll be a number of examples over the years where politicians and ex-politicians have gone on to campuses and there has been a bit of a stir around their views and people using their voice to either protest or support," she said.
ACT party leader David Seymour also chimed in, calling for the resignation of Thomas.
Brash is scheduled to speak in a debate at the University of Auckland on Thursday and as far as the university is aware, the event will go ahead.
Brash has said Thomas had no grounds to call him a supporter of Southern and Molyneux.