His campaign never recovered and National slid to a narrow defeat by Labour.
Dr Brash claims, however, that while the episode might have cost National some votes "at the margin", other factors were far more influential in Helen Clark securing a third term in Government.
These factors included Labour's "electoral bribes" covering interest rates on student loans; the extension of the Working for Families income assistance programme to higher salary bands; a fear campaign targeted at National via an advertising blitz in the last week of the election campaign; plus a buoyant domestic economy.
Dr Brash's memoirs possibly solve one mystery that occurred during his three years at National's helm, but offer no fresh explanation for another. He still does not remember whether he told a group of six visiting American senators that New Zealand's anti-nuclear policy would be "gone by lunchtime" if National won office.
He says, however, that fellow National MP Lockwood Smith, who was at the meeting, is sure Dr Brash used the phrase.
Dr Brash remains none the wiser about who leaked large chunks of his email traffic to investigative writer Nicky Hager and which were the basis of the latter's book on the Brash years, The Hollow Men.
He says the theory that Bill English, the victim of Dr Brash's leadership coup in 2003, was the culprit because his deputy leader's office at Parliament was across the corridor from Dr Brash's office and he would have been able to see when it was unattended was ridiculous, not least for the fact Mr English was never deputy leader while Dr Brash led National.
Extract: Brash on adultery
What of my occasional breaking of my wedding vows? That certainly wasn't part of my Christian upbringing. On the contrary, Jesus is quoted as not only deploring adultery but also saying that 'whoever looks on a woman to lust after her has committed adultery with her already in his heart.'
If that is taken at face value, there is an awful lot of adultery going on. And perhaps that is why devout Muslims require women to be covered from head to toe. It seems to me that the great majority of human males are programmed to find women sexually attractive, and that programming goes way back in evolutionary time.
I am not of course arguing that this justifies my behaviour. In a modern civilised society we all need to learn to behave in ways which involve restraining the biological urges with which we were born. I am simply arguing that there is an extremely powerful biological urge which most men have to learn to control.
The result of that drive is that many men take huge risks for the chance to have a sexual relationship. Among American presidents the names of John Kennedy and Bill Clinton immediately spring to mind, while in earlier years Franklin D. Roosevelt, Wendell Wilkie and Warren Harding would have come to mind. Prominent Christian leaders, such as Martin Luther King, Paul Tillich and of course several televangelists, are also known to have had extra-marital affairs. We're now seeing a depressingly long list of prominent entertainers accused of sexual offences going back decades. Among Hollywood celebrities, it is assumed that both men and women will have lots of sexual partners, and indeed it often seems that that is a prerequisite for being regarded as a celebrity.
I have not had a large number of sexual partners, and none at all before my first marriage. But I have no choice than to admit that the number exceeds the number of women I was married to.
And if lusting after a woman really is equivalent to adultery, I'll be in serious trouble if there is a Judgement Day.
Extract from Incredible Luck by Don Brash on sale today, rrp $40. Reproduced with permission of copyright holders: Don Brash and Troika Books.