Running prisons is expensive - around a $100,000 per prisoner per year. As finance minister, English didn't resent that cost per se, he resented that as a form of investment the return was poor.
He advocated for rehabilitative measures because our prisons had revolving doors.
Nearly three quarters of released prisoner were reconvicted of an offence within five years, and more than half were returning to prison. In many cases, a lag made things worse. Idle prisons become universities of crime.
More importantly, reducing reoffending wouldn't just reduce costs, it would also mean fewer victims of crime. There is perhaps no clearer example of a classic win/win.
On this basis, the government launched a reform agenda, but it hasn't taken the country with it. And this is a mistake. Without building a broad public consensus on the approach's goals and merits, it is at risk from moral panics and knee-jerk politics.
Our base instinct to punish criminals is natural and punishment is an important function of prison.
But often there will be no sentence long enough to placate distressed victims or their loved ones, so as a society we must weigh-up those desires with the interests of the public good.
And here's where we need to start the conversation and build a consensus around the balance between punishment and rehabilitation.
Extending the debate beyond punishment may not be easy. New Zealand has been sold the idea that longer sentences are the solution by both major parties for years.
Penal populists say harsh sentences act as deterrence by making people think twice before committing crime. Deterrence arguments aren't without merit but studies show that likelihood of apprehension is a far greater deterrent than severe punishments.
If extreme sanctions deterred crime we would expect to see fewer crimes in places with the harshest sentences. We don't. In the United States, for example, states that have the death penalty have comparable murder rates than those who haven't.
For reasons of cost and efficacy, then, I can't knock the government's rationale for increasing a focus on rehabilitative policies, but its vulnerability is all too clear.
When Philip Smith escaped to Brazil in 2014, for example, there was an immediate clampdown on temporary releases that allow prisoners to work outside the wire in paid employment.
Despite the escape being a consequence of corruption rather than policy, the Department of Corrections feared a public backlash and restricted temporary releases even though they were widely seen as a successful rehabilitation tool.
This is why the reform policy agenda requires community understanding. By working their reform agenda quietly the government perhaps felt it could avoid the prospect of controversy.
But in reality the public are more likely to turn against an agenda they haven't bought in to. Having removed Judith Collins from the Corrections portfolio, English has very publically signalled that the harder-edged and populist approach does not curry his favour.
But can he change public attitudes in the way Judith Collins and others so cleverly played up to them? Can English convince the country that although the Department of Correction' reoffending target won't be met, its modest gains and the change of thinking it represents is nevertheless the best hope of bringing the prison population down?
I don't know the answer to that; the punitive approach in the New Zealand psyche is so strong we've become one of the most imprisoned populations in the developed world without even flinching.
But I'm pretty sure that, just like English, the vast majority of Kiwis would prefer to spend a billion dollars on things other than a new prison. In this way, it might be economic considerations that motivate the necessary public consensus on the desirability of the approach favoured by the prime minister.