But life for the top 10 per cent has been rosier, with disposable income doubling, he said.
The top 1 per cent owns 16 per cent of all assets, while the bottom half owns 5 per cent.
Maori and Pacific Islanders have twice the poverty rate of Pakeha.
Mr Rashbrooke said inequality causes breakdowns in society, with people becoming isolated and segregated.
"This is one of the greatest drivers of ill health. Countries that are doing better have very low levels of health and social problems.
"Those countries are doing better because people right across the spectrum are doing better."
He said people are told "big global forces" are responsible for the problem.
"Every country in the world has been affected by globalisation, but we've had this world-beating increase in inequality."
He believes there are two causes: predistribution, the initial allocation of income, and the reduced bargaining power of those at the bottom; and redistribution of wealth.
"There's much less redistribution than we're used to."
He said New Zealand needs to boost the bargaining power of those at the bottom, invest in skills and education, and do more to build shared communities. "We don't tax wealth, we don't tax inheritances; we should look more at that."
He said people are beginning to realise how big an imbalance there is.
"This is the single biggest issue for New Zealanders right now. I certainly hope it will be an election issue."
Mr Rashbrooke fielded a question about beneficiary "bludgers".
"Look at the people that queue up for jobs every time a supermarket opens. People want to work. We have to support people in work, rather than punishing them."
Locally, he said people should be questioning employers, in areas where people were badly paid. "Shine a spotlight on it, talk about it. I'm going to do everything I can to make this the number-one issue in New Zealand."
