"They're serious allegations and serious issues, and in such circumstances one would normally be stood down while that investigation takes place.
"He is one of the Government's most senior officials and he gets to stay in his job while this investigation is taking place. I do find that extraordinary.
"It's pretty incredible that we have such discrepancies in what he said, the comments from the Government Communications Security Bureau, and the Finance Minister not being able to express confidence in him [in relation to his advice about the 'hack']."
The GCSB advice to the Treasury was that its computer system had not been compromised, and this was provided before the police were called in and before Makhlouf released a statement saying the Treasury had been "hacked".
Hughes said there were many reasons for someone to be stood down while being investigated.
"In my view, this is not one of those situations," he said in a statement.
"I fully understand that people want to understand what happened here and there is some speculation about that. I can't do that. The investigation will establish the facts of the matter."
The Ministry of Business, Innovation and Employment website says that an employee can be stood down if their presence at work could compromise an investigation by, for example, tampering with evidence.
Another reason is for health and safety, such as if the employee was alleged to have bullied someone.
For example, Retirement Commissioner Dianne Maxwell was stood down from last December until the end of May while claims into bullying were looked at. Maxwell was eventually cleared of the allegations.
National Party leader Simon Bridges has called for Makhlouf to resign for implying that the party had hacked into the Treasury's website to obtain confidential Budget 2019 information.
National staffers had simply used the website's search function, and police have said that nothing illegal appeared to have taken place.
Bridges has also called for Robertson to resign for being "donkey deep" in what happened.
Robertson has said he has no intention of resigning, and has rejected knowing anything about the police referral until Makhlouf told him in a meeting last Tuesday, which took place about an hour after police had already been called in.
Hughes asked for patience so that the inquiry could run its course.
"In the interests of fairness and the integrity of the investigation process, we need to let Mr Ombler get on with the job."