"That's hypothetical. Let's wait and see."
Mr Brownlee has given up the aviation security aspects of his portfolio, but Opposition parties called for him to relinquish the entire transport portfolio until the outcome of the investigation.
But Mr Key rejected this. "I don't think it would actually add much to it."
Labour leader David Cunliffe said it would add "assurance of impartiality".
"It is not appropriate, nor is it credible, that he can be investigated by his own officials. Of course he should be stood down, at least for the duration of the investigation. I would have thought that was an easy decision.
"The Government is more and more looking like an arrogant, out-of-touch government."
Green Party co-leader Russel Norman likened the behaviour to Aaron Gilmore's, who resigned in disgrace after threatening to use his influence with Mr Key and telling a waiter: "Do you know who I am?"
Dr Norman said Mr Brownlee was arrogant to steamroll his way past the staff member and through a door he knew he should not access.
"When he knocked on that door, it was a non-verbal form of 'Do you know who I am?'
"This is exactly the same thing. 'I get to break the rules. I'm the minister'. The minister put him in an invidious situation, where the minister was basically leaning on him to break the rules."
Mr Key said there were no parallels with Mr Gilmore.
"[Mr Brownlee] was waved through by the person in question. Fair enough. Gerry Brownlee takes 100 per cent responsibility for that."
Mr Brownlee did not put pressure on the staff member, he said.
"I've been assured that by the minister and he won't mislead me."
Mr Key said his MPs were not arrogant or complacent, in light of Mr Brownlee, Chester Borrows' speeding ticket, and Claudette Hauiti's wrongful spending of taxpayers' money.
"That is a narrative that the media might want to create, but it's actually not the reality.
"From time to time people make mistakes. [Mr Borrows] was 1km/h over the limit. It was unacceptable, but not excessive. Claudette didn't understand the rules.
"I think it shows that they're human beings and they make mistakes from time to time."