Sure, Greg agreed, mandatory minimum sentences and the death penalty seemed extreme.
But he cut me down from my high horse. "You know, New Zealand's not so great."
Prison is a profoundly archaic concept.
Don't like what someone does? Lock them in a cell.
No one is suggesting our most dangerous criminals should be free to roam the streets, but as the previous Prime Minister put it, building more prisons is a moral and fiscal failure.
New Zealand's incarceration rates – especially those of Māori - should be a source of greater national shame.
A few quick fixes should help and banning private prisons is a good place to start.
There are few things more democratically offensive than a government overtly incentivising corporations to incarcerate its own citizens.
Scrapping the senseless three-strikes law is another good move and a sentencing council makes sense.
But Andrew Little and the Government still face a daunting challenge if they're to achieve their goal and sensibly lower the prison population by 30 per cent over the next 15 years.
First of all, they need the next Government's support. And they'll need to continue pouring resources into mental health and addiction services.
But the problem is a single one-off horror story can massively derail reform.
All it takes is one mistake.
The Parole Board will have a good idea of the challenge Andrew Little faces. One offender, mistakenly remanded on bail has the power to completely shift public opinion with a single heinous crime.
Sadly, it'll probably happen. It'll be politicised.
And it's up to us to look back at this point, and remember that before any justice reforms; we weren't getting the balance right.