Asked whether he could get support for the reforms from coalition partner NZ First, which has a tougher stance on criminal justice, Little said: ""People will see there is a better way of dealing with offenders across the spectrum.
"Of course there will be the hard end for whom the prison system is the best place for them to be.
"But we know that the majority of prisoners offend because of a whole heap of other reasons which if we address and work with them on, we stop their offending and the community is safer as a result."
The biggest contributor to the rising prison population – which comes when crime is falling – is a law change in 2013 which made it harder to get bail.
The availability of electronic monitoring and lack of housing for bailed prisoners was also contributing to the rising muster.
About 200 prisoners cannot be bailed because there is no suitable housing for them to go to.
Little said it appeared that prisoners were struggling to get approved for social housing.
"That creates a problem where for some people the only stable household that you go to will be a Housing NZ house," he said.
A spokeswoman later clarified that there was no ban on prisoners going into state houses, but that they had to meet criteria set out by the Ministry of Social Development.
National leader Simon Bridges said he was against the softening of bail laws or measures that could make the public unsafe.
He claimed the Government was looking to progress legislation soon to repeal three strikes laws, which he said would also create a risk to the public.
One of the few people to be sentenced for a third strike, Dylyn Davis, appeared in court today in Hamilton. He was first convicted for wounding with intent to injured, then later for aggravated robbery, and today for murder. However, he escaped the maximum sentence without parole and was jailed for 20 years.
When it was pointed out that Davis would have been given a long sentence anyway because his third strike was a murder charge, Bridges said the legislation gave the judge the option of going further than the usual sentence.
Asked whether a prison population of more than 10,000 was acceptable, Bridges said: "Of course it's not.
"No one likes prisons, but this is the least bad outcome. And what we don't want to see is New Zealanders unsafe.
He said people were only locked up for serious offending.
"We're not talking about theft of a Moro bar. We're not talking about the lowest lower end [crimes]. We're talking about very serious cases."
The three strikes rule is an Act Party policy that was adopted by the National-led Government in 2010, and applied to 40 violent and sexual offences.
It ensured that third-time offenders received the maximum sentence and no parole.
Ministry of Justice data shows that so far this year, 348 people have received a first warning for these offences, and 16 people received a second or "final" warning.
