"As parliamentarians you might say we become inured to all of this, although I doubt it."
He paused several times to compose himself, though at one stage joked he could use some gin, prompting laughs.
First elected as MP for Griffith in 1998, Rudd became prime minister in 2007, defeating John Howard.
But his time as prime minister came to an abrupt end in June 2010 when he was dumped by his party in favour of Julia Gillard.
Rudd reclaimed the job of prime minister in June this year when he defeated Ms Gillard in a party-room showdown.
But he went on to lose the September 7 election to Tony Abbott.
Mr Rudd thanked the "good burghers of Griffith", saying they were a good people and community.
"The decision that I have made has not been taken lightly, particularly given the big attachment I have for the community I proudly represented in this place these past 15 years," he said.
He also thanked the people of Australia for electing him as their prime minister.
"To have served as prime minister of Australia has been a great honour afforded to very few in our country's history," he said.
"For the future I wish the prime minister and this government well. I do that because I wish Australia well.
"The prime ministership of this Commonwealth is not easy; it is the hardest job in the land.
"The expectations of whoever holds the office are infinite while the resources available are finite."
He also wished Opposition Leader Bill Shorten all the best in that role.
He thanked Anthony Albanese for his "extraordinary service" as deputy prime minister, and Chris Bowen for his "great contribution" to the Australian Labor Party.
Mr Rudd said as prime minister he'd navigated Australia through the great financial crisis without a recession or mass unemployment, and maintained the nation's AAA credit rating.
His government had introduced a national curriculum, first ever paid parental leave scheme and a pension increase.
But it was the official apology to indigenous Australians in 2008 that remained the highlight of his political career.
"Nothing has brought me greater joy in political life than the smiles I have seen on the faces of our Aboriginal brothers and sisters, young and old, country and city, as a result of the apology," he said.
"I hope though that what we've archived through some healing of the soul will be the first step, the second of course is closing the gap to achieve a healing of broken bodies as well."
Mr Rudd said it was a privilege to be asked to return to the prime ministership this year, and that they'd been able to "save the furniture" in the election.
"In fact I think do considerably better than that," he said, to cheers of "hear hear" from the opposition frontbench.
"I'm glad that all you folk in the frontbench were returned in one piece as well, and that we return Labor as a fighting force for the next election."
He said the internal reforms made to the ALP this year were a "great experience for the party", but should only be the first step.
"In the meantime I look forward to a full democratic preselection process for all local party members to elect our next candidate for Griffith," he said.
Mr Rudd will establish a national apology foundation and will support causes for homelessness, organ donation, and multicultural issues including foreign language education and interfaith dialogues.
He intends to be active in the international community where he can make a contribution to peace and stability, global economic governance, sustainable development and climate change.
He thanked parliamentary staff and the press gallery, who he encouraged "to apply all necessary programmatic specificity to the task of holding the government of the day to account" - referencing one of his oft-quoted peculiar bureaucratic phrases.
"Whatever has been said, and a lot has been said that's hurtful ... I bear no one in the place any malice," he said.
"Life is far too short for that.
"It is time however for the baton unequivocally to be passed to others."
He thanked his family and wife of 32 years, Therese Rein, and asked for privacy once he's no longer a public figure.
"I'm not planning on any interviews any time soon," he said.
Mr Rudd said Australia was a great land of opportunity.
"To think that the son of a dairy farmer whose family who didn't really have much money could secure a place at university through the Whitlam reforms and upon graduation become a diplomat and then serve as chief of staff to a premier, be elected to the parliament to represent the great Australian Labor Party and ultimately to be elected as leader of the party and then as prime minister says everything about how extraordinary this country is," he summarised tearfully.
Mr Rudd finished his speech with his now-trademark line: "It really is time for me to zip."
- AAP