But Wilders still gained seats, reconfirming his role as a sharp thorn in the side of the nation's more centrist leaders.
"Rutte is far away from rid of me!!" Wilders wrote on Twitter shortly after the initial exit polls were released. He appeared to acknowledge that he had not bested his rival.
Addressing an election night gathering of supporters, Rutte said "the Netherlands said 'Whoa! Stop!' to the wrong kind of populism". Rutte added: "We want to stick to the course we have - safe and stable and prosperous".
Taken together, the initial results appeared to show a nation that agreed that it disliked the status quo - but not about an alternative direction.
The ruling centre-right Party for Freedom and Democracy remained the largest party in the exit poll, but it was on track to lose nearly a quarter of its seats in Parliament, forcing Rutte to broaden his coalition across the political spectrum. His coalition partner, the centre-left Labour Party, may be wiped out as a political force, plummeting in initial forecasts from 38 seats to nine out of a total of 150.
Even as Wilders confronted limits to his ballot-box appeal, his agenda-setting power remained evident after many mainstream politicians tacked rightward during the campaign to advocate for stricter limits on immigrants.
His Party for Freedom was forecast to build slightly on its current 15 seats in the lower house of Parliament, tying it with the centrist Democrats 66 party, and the centre-right Christian Democratic Appeal. The centre-left Green-Left party also appeared to do well, potentially quadrupling its seats.
The likely formation of a broad, weak coalition across the political spectrum could give extra ammunition to Wilders even if he is shut out from power. Rutte has repeatedly said he would not work with the peroxide-haired firebrand.
Rutte also significantly toughened his stance on immigrants during the campaign in a bid to capture Wilders's supporters, telling immigrants in January to "act normal or go away".
But the Wilders' showing will still probably slow the momentum of French anti-immigrant leader Marine Le Pen, who, if she captures her nation's presidency in May, would try to lead France out of the EU, shattering the bloc in the process.
German leaders also face a challenge as the anti-immigrant Alternative for Germany party stands to capture seats in Parliament.
The initial results were greeted with relief outside Dutch borders.
"Large majority of Dutch voters have rejected anti European populists. That's good news," the German Foreign Ministry posted on its official Twitter account.
The tone of the Dutch campaign dispirited some citizens who want a welcoming attitude toward refugees and immigrants.
"They're not making the point what they want to do. They're just saying what they're against," said Arieke Maljaars, 32, a teacher at a school in the heavily immigrant Schilderswijk area of The Hague, where Turkish kebab stands are close to Surinamese grocery stores.
She said she planned to vote for the small, centrist Christian Union party. She said some of her 8- and 9-year-old students, most of whom are immigrants or their children, "were really scared".
"One of them said, 'Maybe I'll have to go to Turkey, and I really don't want to go there.' For children in the neighbourhood, it can feel frightening," Maljaars said.
In a final debate, Wilders attacked his opponents for allowing in too many immigrants.
"Every day we're confronted with the mess of this," Wilders told deputy Prime Minister Lodewijk Asscher of the centre-left Labour Party. "This is your scum. I hope you'll learn lessons, because to what has your policy led?"
The campaign has upended old notions about Dutch tolerance and inclusiveness, and the influx of immigrants starting in 2015 has created the perception of new strains on society. There was a net increase of 56,000 people in 2015, and 88,000 in 2016, many of them Syrian, according to the Dutch Central Bureau of Statistics. About 10 per cent of the Netherlands' 17 million people are non-Western immigrants or their children.
The refugee crisis has helped fuel the Dutch debate. The push-and-pull comes at the same time as Trump's triumph in the United States, the British rejection of immigrants after the decision to leave the EU and the fiery immigration-focused election campaigns in France and Germany.
Many Wilders supporters said today that they resented that refugees who came to their country were provided housing and health care even as Dutch people struggled to make ends meet.
"I understand they don't have anything, but I have to pay for all that," said Bep van Beele, 66, who lives in the working-class Duindorp area of The Hague, a bastion of Wilders's support. "It creates jealousy. There's not much left for the Dutchman."