"The Greek people have spoken", said Mr Samaras in a televised statement. "Everyone respects their decision. My conscience is clear."
To the booming sounds of Rock the Casbah by The Clash, Syriza's leader Alexis Tsipras told a rally of thousands of supporters he would defeat "austerity which destroys our common European future".
"I would like to reassure you that the new Greek government will be ready to co-operate and negotiate for the first time with our partners for a mutually beneficial and sustainable solution so Greece comes out of a vicious circle of debt," he said.
"We have a great opportunity for a new beginning both in Greece and in Europe. For a new policy, for a new model of relations based on mutual respect.
"Our priority from tomorrow will be to restore popular sovereignty in the country, to give justice, to clash with old establishments. To clash with the regime of corruption. To promote reforms in the state, public administration, everywhere.
"Our priority above all will be to restore the country's lost dignity. We regain hope, we regain smiles, optimism and dignity for our people."
The outgoing Prime Minister was unrepentant. He said: "I received a country which was almost destroyed and I was asked to take the hot potato and I did that.
"Most people didn't give any prospects that we would endure... We had to take difficult measures and there were some mistakes and injustices but we averted the worst.
"I am handing over a country that has no deficit, secure for the citizens... a country that gets out of the crisis in an organised way.
"I wish sincerely that my predictions do not come true, but I had to warn everyone."
Syriza is led by the 40-year-old Mr Tsipras, who looks set to be Greece's youngest Prime Minister for 150 years.
He is known for his relaxed attitude, travelling by motorbike and preferring open-necked shirts to a suit and tie.
He lives in an apartment block in a working-class suburb of Athens with his partner and two children - the youngest of whom has the middle name Ernesto after revolutionary Che Guevara.
His party wants to renegotiate the terms of Greece's 240 billion euro bailout with the EU and the International Monetary Fund.
It says repayments are stifling Greece's chances of recovering from a six-year recession - but Syriza's popularity has spooked markets which fear a new financial crisis could push Greece out of the euro.
Syriza party spokesman Panos Skourletis said it was "a historic victory that sends a message that does not only concern the Greek people, but all European peoples
"There is great relief among all Europeans. The only question is how big a victory it is."
Left-wing French president Francois Hollande phoned the party leader to express his "desire to pursue the close co-operation between our two countries in service of growth and the stability of the euro zone, in a spirit of progress, solidarity and responsibility that is at the heart of the European values we share."
Tied for third place on 6 to 7 per cent of the vote in exit polls were the centrists To Potami and the far-right party Golden Dawn, which critics have slammed as being neo-Nazi and racist.
Prime minister Samaras had already urged voters not to push Greece to the "brink of catastrophe" in today's crucial general elections.
He defended harsh austerity measures in a speech to fellow members of his conservative New Democracy party, saying he had no choice as the "ship was sinking".
The apparent Syriza victory means a eurozone crisis could now form the backdrop to a second successive General Election in Britain.
Tory Party chairman Grant Shapps raised the prospect that Britain could end up in similar to trouble to Greece if Labour wins power and fails to tackle the deficit.
He said: "Five years ago Britain had the same sized deficit, as a proportion of our economy, that Greece had. We took one route, which was to sort out, take the difficult decisions and sort out our economy. Greece has not done that and you see the chaos they're in."
But former Labour Cabinet minister Peter Hain welcomed the prospect of a Syriza victory, saying it would reopen the debate about austerity.
Mr Hain, a close ally of Ed Miliband, said: "I hope Syriza wins because it will be a big kick to the orthodoxy - the austerity - gripping most of Europe and most of the world, including Britain."
Today's general election was held almost two years early and was crucial for Greece's financial future.
Mr Samaras had appealed to undecided voters to ensure Greece stayed on the path of stability and reform.
"Today we are deciding if we move ahead with power, safety and confidence or if we get into an adventure," he said after voting in the western Pelopponese region.
"I am optimistic because I believe no-one will risk the European course of our country."
But opinion polls also showed a significant portion of Greek voters were undecided two days before the general election.
Speaking as the first ballots were cast on Sunday, Mr Tsipras said Europe must find an alternative to austerity.
"Our common future in Europe is not the future of austerity," he told reporters after casting his vote in Athens.
A Syriza win would represent another turning point for Europe after last week's announcement by the European Central Bank of a massive injection of cash into the bloc's flagging economy.
The bank has spent years trying to clamp down on budgets and pushing countries to pass structural reforms.
Some 9.8 million Greeks were eligible to vote. Final polls on Friday gave Syriza a lead of up to 6.7 points with 31.2-33.4 per cent of the vote, close to the level needed for an outright victory.
After its most severe crisis since the fall of the military junta in 1974, Greece's economy has shrunk by some 25 per cent.
Thousands of businesses have closed, wages and pensions have been slashed and more than half its young people are unemployed.
At the same time, its massive public debt has climbed from 146 per cent of gross domestic product in 2010 to 175.5 per cent last year, the second highest in the world.
The country's creditors insist Greece must abide by its commitments to continue receiving support, and investors and markets alike have been spooked by the anti-bailout rhetoric.
Greece could still face bankruptcy if a solution is not found, although talk of 'Grexit' - Greece having to leave the joint currency - and a subsequent potential collapse of the euro itself has been far less fraught than during the last general election in 2012.
The Prime Minister's campaign focused on the gradually improving economy, which grew for the first time in six years in the third quarter of 2014, and has promised to reduce some taxes if re-elected.
He has warned of the potentially dire consequences of reneging on bailout conditions - to the point that his critics accused him of running a fear campaign.
But Syriza's promises of ending the crushing austerity Greeks have been living under since 2010 have wooed many voters infuriated by the deterioration in their standard of living and ever increasing tax bills.
The big question is whether any party will win the required 151 of parliament's 300 seats to form a government alone.
The Greek political scene has fractured during the financial crisis, with voters abandoning the two formerly dominant parties - the conservatives and the socialists - in favor of a smattering of smaller parties.
In their final day of campaigning on Friday, both leaders appealed to the undecided voters, which opinion polls put at around 10 per cent in the days before the election.
Without the required 151 seats, whichever party wins will have to try forming a coalition government with another party. The first three parties each have three days to try and form a coalition government to avoid a second election being called within a month.
Another option, however, would be for the winner to seek support for a minority government, where other parties would vote along government lines without participating itself in a power-sharing deal.
Opinion polls ahead of the vote showed the new centrist Potami, or River, party vying for third place with Nazi-inspired Golden Dawn, whose leader and several top lawmakers are in jail awaiting trial on charges of participating in a criminal organization.
Whichever government emerges, it has a series of formidable tasks ahead of it, the most pressing of which is concluding frozen negotiations with bailout inspectors to release a 7.2 billion euro (NZD $10.8 billion) loan installment originally due late last year.
The inspectors "must come soon", Finance Minister Gikas Hardouvelis said Saturday.
The new government will also have to negotiate some kind of relief for Greece's 320 billion euro debt and bolster weak growth.
- Daily Mail
