Swan opened his attack last week with an article in the political affairs magazine The Monthly and continued it yesterday in a speech to the National Press Club in Canberra, in which he also accused Opposition leader Tony Abbott of "singing for his supper".
Abbott has defended the billionaires, who Swan said were large Liberal Party donors.
Swan said the "ruthless individualism and unquestioning materialism" of the super-rich were undermining the traditional spirit of fair go in a nation that prided itself as being more equal than most.
"To be blunt, the rising power of vested interests is undermining our equality and threatening our democracy," he said.
"The infamous billionaires' protest against the mining tax would have been laughed out of town in the Australia I grew up in, and yet it received a wide and favourable reception two years ago.
"A handful of vested interests that have pocketed a disproportionate share of the nation's economic success now feel they have a right to shape Australia's future to satisfy their own self-interest."
Swan said that while Australia was now mainly a middle-class society, there were large pockets of considerable social disadvantage and there was a lot still to be done to tackle poverty.
He said allowing vested interests to distort the shape of economic growth for their own narrow advantage was bad for democracy, the community and the economy.
The market system was still the best mechanism for generating prosperity for more people, and Australia could not afford to let it be undermined by the "excessive greed of a wildly irresponsible few".
Swan said that in the past few years coalitions of the rich had emerged, using their considerable wealth to oppose good public policy and economic reforms designed to benefit most Australians.
"The combination of industry deep pockets, conservative political support, biased editorial policy and shock-jock ranting has been mobilised in an attempt to protect vested interest," Swan said.
"There has been a perceptible shift in this country in recent years, and it is sadly very much in the American direction of stronger and stronger influence being wielded by a smaller and smaller minority of vested interests.
"Crucially, much of our media seems more and more inclined to accept that growing influence."
He said the latest example was Rinehart's Fairfax foray: "Without a blush, her friend and fellow media owner John Singleton let the cat out of the bag when he told the Sydney Morning Herald that he and Rinehart had been 'able to overtly and covertly attack governments ... because we have people employed by us like [columnist] Andrew Bolt and [talkback hosts] Alan Jones and Ray Hadley who agree with [our] thinking'."
Swan said Australia's success had never been in more jeopardy because of the rising power of vested interests, a "poison [that] has infected our politics and is seeping into our economy".
Palmer retaliated with a broadside in Fairfax newspapers, saying that Swan's attack was "the mark of an intellectual pygmy" and the classification of people by their means, race, class or gender was no substitute for robust discussion about ideas or solutions to national problems.
"It is common Labor Party practice to attack others rather than to examine the shortcomings of Labor policy or Labor leaders," he said.
Forrest's Fortescue Metals Group took out full-page advertisements against Swan's "unfounded attacks" in major newspapers yesterday, saying the Treasurer was trying to polarise and spread resentment in Australian society.
The group said it would pay more than A$1 billion in taxes, royalties and other government assessments this year - in addition to billions spent on infrastructure and the creation of thousands of jobs - and that Forrest had given hundreds of millions of dollars to charities.
Abbott said Swan's claims were "half-baked neo-socialism" and that the big miners had created jobs, investment and prosperity.
"They are wealth creators," said Abbott. "Wayne Swan is a wealth waster."
