Mr Brown said that equated to the headline 'Serious burglary, nothing missing.'
"That's what it is," he said.
"I'm brave enough to speak out, but a lot of people aren't. And a lot of people have been hurt."
Former chief executive David Edmunds had been "pushed out" over something that didn't exist. He had no idea of what had become of Mr Edmunds, but the chief financial officer had left and had been unable to find a new position.
"This has upset a lot if people who can't find new jobs," he added.
"Nobody has approached me over the last 18 months. No one wants anything to do with anyone who is even vaguely implicated in fraud.
"John Carter is a hell of a good politician and a really good bloke, but he doesn't understand numbers. I"m not as good a politician as he is but no one has ever questioned my financial acumen."
And he was not prepared to take Mr Carter's word that the SFO inquiry had been good value for money, until he knew what it had been asked to do and how much it had cost.
"Tell us that and we will decide if it was good value," he said. "If [Mr Carter] carries on with this bullshit about good value for money he won't get rid of me."