The Liberal Party, believing it had little chance of attracting those voters, focused its attention elsewhere. In recent years, things have changed.
Second and third-generation migrants are better educated, wealthier and more aspirational than their forebears.
The voting patterns of long-established Greek and Italian communities resemble those of everyone else. And the entrepreneurial Chinese, Filpinos and Indians who dominate more recent arrivals are not Labor voters.
Salma Khan, a Pakistani-Australian whose family owns two 7-Eleven franchises, says: "Before, we were in Labor because my husband was working in a factory ... [But] the Liberals are better for business."
The Liberal Party has woken up to this new constituency, particularly in New South Wales, where ethnic support - including that of Lebanese Muslims, traditionally Labor voters - was decisive in Barry O'Farrell's victory at state level in 2011. For the federal election, the Liberals are fielding an impressive cast of migrant-background candidates, particularly in multicultural western Sydney.
Yat-sen Li, a high-flying corporate strategist who speaks Mandarin and Cantonese, was born and brought up in suburban Sydney. However, he had no connection with Bennelong until July, when Labor dumped its original pick, Jeff Salvestro-Martin, after he was drawn into the New South Wales corruption inquiry.
Rudd asked Yat-sen Li, who was running his own consultancy in Beijing, advising Australian companies on doing business with China, whether he would step into the breach.
Within days, the 41-year-old had moved back to Australia, where he bought a house in the electorate.
Accusations that he was "parachuted in", in an effort to woo Asian voters, are difficult to refute. And the mostly well-educated, well-off Asian voters of Bennelong can see through such stunts.
Moreover, the sitting Liberal MP, the former tennis star John Alexander, whose majority is 3.1 per cent, has three years of community bridge-building under his belt.
The Liberals - whose social conservatism also appeals to many migrant voters - have been steadily making inroads into Labor's territory. At local government elections last year, they seized control of a swag of western Sydney councils - and many of the new councillors and mayors are from non-English-speaking backgrounds.