"A lot of the gains have come from frontline staff - seeing that things can be done better and having the opportunity to get on and prove it.
"And we expect that we can continue to make gains right through the next three, four, five years," he told the Herald.
Prime Minister John Key emphasised the patterns of spending in his pre-Budget speech in Wellington last month - a dramatic arrest in the rate of increase.
His office had calculated that the net new discretionary operating expenditure in the last five years of Labour had been $19 billion compared with $1.6 billion for National's, including Thursday's Budget - after demand-driven rises such as increases in superannuation and benefits had been removed.
"That's a hell of a lot of tax revenue," he told a business audience.
Much of the public attention in the public service goes on the regular announcements of job losses - the most recent being from the Department of Conservation, although the initial plan to cut 140 jobs was amended to 72, and Friday's announcement of 30 new biosecurity staff has led the Public Service Association to question the loss of 90 over four years.
The latest Cabinet paper on the staff cap forecasts that by June this year, the core Government administration cap will be at 35,839 - 3020 fewer than when the cap was set in 2009.
Mr English says that while there have been imperatives to save money especially on what he terms "back office costs", some important changes have been made in the way Budgets affect the public sector.
The first was the decision in 2010 to set and pretty much stick to a four-year spending plan.
Part of the rationale for that was to get away from "spending three months of the year squabbling with the Treasury" over the detail about an extra $2 million or so and to get them spending more time on their longer-term strategy, how to improve services, and how to improve their organisation. "It's working."
The Government was willing to put more money into an organisation in which it had faith that the money would be used wisely.
He said one of the biggest steps was the announcement last year of the 10 targets under the Better Public Services plan, such as lowering violence against children, reducing reoffending and getting more students through NCEA level 2, over five years with regular public reporting on the targets and an expectation that departments worked together on the targets.
"We are not really squeezing them ... what we are finding is when they focus on actually achieving lower re-offending rates or less abused children or more success in schools, that's what saves us money."
If any of the prisoners who are about to be released from prison in the next few years were sent back to prison, that would cost $900,000 a year, and a further $250,000 in capital if a new prison bed was needed, he said in a speech this year to the Institute of Public Administration.
The management of spending in the justice sector is handled differently with ministers able to shift money around between Budgets to where they deem it is needed.
The Probation Service got a boost in spending when National gained office in 2008 and was exempted from the public service staffing cap (as were Work and Income and Child Youth and Family).
"Since then," says Mr English, "they have made considerable savings which they are redirecting through the justice sector to reduce reoffending rates."
In social welfare a lot more money had been spent on trying to reduce welfare dependency. "As those numbers come back, that will save us a lot more than we put in."
Public Service Association national secretary Brenda Pilott says the PSA does not believe there "is fat to be cut out of the system - to use that language".
Departments were under pressure to fund wage increases and now employer KiwiSaver contributions from their votes and collective agreement settlement had been modest and reasonable.
"I think that reflects two things: one is understanding that departments don't necessarily have the money to pay and the other is that at times of high unemployment and a lot of restructuring people are very concerned about having a job and that does tend to depress the wage claims."
Former public sector chief executive Len Cook believes big changes are ahead in the next decade in the public sector driven by changes in technology and demographics. But more planning needed to be put into how to manage the change without the "over-restructuring" that had occurred. "I think the big issues is knowing how to manage it," said Mr Cook, a former government statistician and past president of IPANZ.
Public service staff
* Cap March 2009: 38,859
* Cap after reset in March 2011: 36,475
* Actual December 2012: 36,081
* Forecast June 2013: 35,839
* Govt cap applies to core Govt administration staff, full-time equivalents.
* Cap excludes Prison and Probation Service, Work and Income, Child Youth and Family.
* Includes Housing NZ, Qualifications Authority, Trade and Enterprise, Transport Agency, Tertiary Education Commission.