He rose through the ranks at the union, eventually becoming national secretary, and helped the organisation grow into one of the largest in the country with around 40,000 members.
After Labour's 2008 election defeat, Mr Little became party president.
He ran for Parliament in the New Plymouth seat out of concern about the erosion of workers' rights, such as National's introduction of the 90-day probation law.
Elected on the list in 2011, he was not expected to remain on Labour's backbenches for long.
He gained some profile as the party's ACC spokesman when the corporation went through significant changes under National.
His attacks on then-minister Judith Collins during a mass privacy breach at ACC prompted the minister to take defamation action against him. He and Trevor Mallard had accused Mrs Collins of leaking an email, but the case was settled out of court.
Mr Little moved up Labour's list in a caucus reshuffle in 2013 and took on the justice and labour portfolios.
In another reshuffle early this year, former leader David Cunliffe promoted him a further six places to number 11 because of the "heavy lifting" he had done in those portfolios.
Mr Little lobbied for a reversal of legal aid cuts, and sought the introduction of corporate manslaughter laws to increase accountability in cases such as the Pike River disaster.
He helped develop new policy which improved the rights of sexual assault victims and reversed the burden of proof in rape cases, though the policy's release was overshadowed by Mr Cunliffe's "sorry for being a man" speech.
Mr Little also won attention for performing the Gangnam Style dance in the House while accusing National of "riding the invisible horse of decision-making".
App users: Watch 'Andrew Little - Gangnam Style Government' here
Now living in Island Bay in Wellington, the 49-year-old is yet to make an impression in the New Plymouth electorate.
He has run two failed campaigns in the seat, with National MP Jonathan Young's majority growing from 4000 votes in 2011 to 10,000 votes in 2014, partly influenced by a boundary change.
After scraping into Parliament in the preliminary election results last month, he had a nervous wait as the special votes were counted.
When his re-election for a second term was confirmed on Saturday, he immediately set his sights on the leadership.