"So I took a step back from that and said 'yeah, okay, well fine'. That's the way it goes."
The pair were talking about how they operate together and with Cabinet and where the points of difference have been.
Mr Key also admitted he had been very reluctant at first to raise GST in 2010 but was persuaded fairly quickly about its merits.
"I'd be the first to admit I was a bit nervous about raising GST thinking can you actually politically sell all of that," he said.
"Actually after we did all the modelling and we worked on it together, we were absolutely convinced it was fair and would actually work and it would deliver the sort of policy outcomes we wanted. And actually it's definitely delivering results for the economy."
The pair said they did not have arguments or rows.
Mr Key said the measure of any decent relationship was that you worked your way through all sorts of issues and respected each other's views.
Mr English made much of what he described as Mr Key's instinctive ability to communicate with the public and maintain its support, and knowing how to set boundaries in terms of policy constraints.
They cited the example of state tenants' entitlements.
Mr Key said successive Ministers of Housing and Housing officials had wanted the income that any state tenant received from boarders to be received to be counted as income in terms of calculating entitlements.
"But my view is well that would be seen as a step too far for large families or families that are trying really hard to make ends meet.
"And in the end if they are prepared to go the extra mile of having someone live in their home and cook them a meal, they are just good New Zealanders trying to get ahead.
"It's like the carparking [dumped fringe benefit tax] issue.
"In the perfection of the IRD officials, we should have carried on with putting an FBT on those carparks - but that's how you lose the public," he said.
Mr Key also indicated that he had put constraints on labour market reforms.