"I don't beat myself up over that stuff because it has just come as part of the job...So we had a view Bronagh was much stronger on it than I was - that I would be running away, and why would I do that?"
That conversation haunted me, and when the Herald featured it on its front page the day the book was published, he told a press conference he had been concerned only for Bronagh. But there was more to it. He was not going to let this job consume him.
Four years further on, nobody can seriously accuse John Key of running away. He is at the height of his success, a third-term Prime Minister suffering none of the frustrations of "third-termitis". The country is not tired of him, as it has been of every other Prime Minister to survive this long. Helen Clark once warned him the third term is dreadful. But he and his Government are still polling at 50 percent and a fourth term looks assured if he wanted IT.
He doesn't. He has had enough. His job is not his life and never has been.
The devastation his party and all his supporters must be feeling since the announcement yesterday is probably much like his colleagues at Merrill Lynch were feeling 15 years ago when he told them he was giving up the financial heights of London and Wall St go home and try his hand at politics.
John Key gives people so much confidence in his leadership that it is possibly dangerous. Key has enjoyed the confidence of business to a greater degree than any Prime Minister since New Zealand became an open economy, and for good reason. His international business credentials are probably better than anyone in business here.
The worry now it that he may be thee main reason New Zealand has been an island of economic stability and political contentment in a world of uncertainty since the global financial crisis. If Key believed that, would he be stepping down?
He would put the country's interest ahead of his personal inclinations, I think. I hope. But he wouldn't put it ahead of the interests of his family. He kept his children well away from the public when they were at school. That has become harder to do now that they are young adults and I suspect he and Bronagh worry about that.
When it comes to the country's interest, his decision to walk away at this point could be called tough love. If the strength of the economy is depending too much on business confidence in him, it would be better if it did not. He leaves with the Government in good shape, the public accounts heading into healthy surpluses, unemployment low and the population rising on a migration wave.
If New Zealand can sustain this strength and confidence in itself without him at the helm, his achievement will be complete.
• John Key, Portrait of a Prime Minister, was published by Penguin Books in 2014.