He also would not commit to keeping super universal, saying the principal aim was a fair and sustainable scheme. "It's an issue we have to deal with and we are going to look at it."
However, within hours Mr Little had backtracked and a spokeswoman said any suggestion of means testing was "off the table".
Mr Little went to ground after making his initial comments and refused further interviews to clarify it. Although Mr Little was answering direct questions about means testing at the time, the spokeswoman denied he had talked about it.
"It was never on the table, is off the table and he wasn't talking about means testing in any form."
The issue overshadowed what was supposed to be a big day for him, eclipsing his response to the Government's Budget.
Mr Little also had to slip in an apology for a quip in his speech about "fiscal gender reassignment" which had offended some.
Mr Peters, who receives super as well as his MP's salary, said it was a "nonsense" to suggest removing super from people who paid taxes their whole lives.
It was also politically fatal to touch super as both National and Labour had discovered in the past. "If they haven't learned that political lesson, then let them have a go."
Mr Peters said many worked because they could not afford to retire on super alone.
"There are a whole lot for whom retirement is not a choice."
He had no qualms about taking it himself. "I have been working since I was six or seven years of age."
About 22 per cent of those aged over 65 work - up from 15 per cent in 2007 and 7 per cent in 1999.
Terry King, Grey Power's national president, said he was concerned superannuation was again becoming a "political football" after comments by both Mr Little and Act leader David Seymour.
"It is important to understand that national superannuation was not a benefit, but an entitlement which most recipients have paid into for 50 years of their working lives. It was theirs by right, not by privilege or charity."
Mr Peters said those who wanted to cut super should target the 70,000 migrants who qualified for it after 10 years of living in New Zealand, irrespective of whether they ever paid tax.
Prime Minister John Key said it would be "grossly unfair" to withhold the pension from those who were still working.
"A lot of them, the lowest income New Zealanders with the lowest amount of savings, the combination of work and their pension gives them a much nicer lifestyle in retirement."
Labour is reviewing its superannuation policy after campaigning to increase the age to 67 for the past two elections.
Mr Little said he believed the age should stay at 65 but something had to be done to address the costs, including starting contributions to the Super Fund again.
Act leader David Seymour also said it was one of the biggest failings of the Budget and proposed a referendum on the issue.
Mr Key denied the current scheme was unaffordable, saying National was managing to reduce debt without trimming super.
