However, there have been similar murmurings around the parliamentary Press Gallery and other political traps that something has changed in the persona Key presents to the world and that his demeanour suggests someone now less than enthralled with some aspects of his job.
This is all grist to a lingering suspicion that Key - despite his repeated assurances to the contrary - will not see out the current term as Prime Minister and will bow out some time before the 2014 election.
This is guesswork based on nothing more than supposition that he is not as addicted to power as a Helen Clark or a Jim Bolger who would have stayed on into their dotage if allowed.
But it is in Labour's obvious interests for such a notion to take hold among the wider public.
The danger for Key is that if people start thinking he is tiring of the role they will claim to see evidence of that when it is simply not there.
Those outside the Beehive who speak regularly to him say Key is just as consumed by and focused on the constant ebb and flow of politics as he has always been. There was evidence aplenty of that last Tuesday. With National predicted to face a torrid time on the first day of the new parliamentary year, Key prepped himself to the maximum beforehand, came down to the House primed for a fight and brushed aside Labour's tepid challenge with ease.
If Key is acting differently - and you would be hard pressed to notice much difference - it may be down to two things.
First, his honeymoon with the media finally ended abruptly with the "teapot tape" saga. His trust of the media was badly shaken; the media saw a different, less attractive side of him.
Second, things invariably start to go wrong in a Government's second term. The year has not got off to a good start. Key and his ministers have had to grapple with the current Treaty wrangle and the Maori Party's threat to walk out of the governing coalition, yet another frustrating Waitangi Day, the public's unhappiness with the Crafar farms decision, revision of Budget surplus predictions and the fuss surrounding his electorate chairman's presence on the board of NZ on Air.
It is such irritants that start to erode the credibility and the popularity of a Government. Key must also be on strict guard for signs of "second termism" - ministers believing in their own omnipotence and wisdom being the only course of action. There has also been something of a vacuum. Insiders say ministers are frantically busy with what is an extremely heavy work programme.
Until the detail is sorted out and policies announced, however, National cannot set the political agenda. It is comforted by the fact that Labour's new leadership is not doing so either.
David Shearer sees no need to rush his revamp of Labour. The pressure will start to build for him to show his hand.
But he has time. Key does not. He is facing an exponential increase in public expectations that National delivers much more this term, particularly sustained economic growth.
Key has rejigged his Cabinet accordingly, handing Steven Joyce important roles in economic development, job training and job creation. There are high hopes that Hekia Parata will shake up the schools sector.
But in highlighting National's 120-point economic plan, Key has stressed there is no magic bullet. It is all going to be hard slog - and made harder by the uncertainty surrounding the European and United States economies.
Worse, even if there were easy answers, they would likely require money to implement them and National simply does not have that commodity on hand.
This year's Budget will be even tighter than its skinflint predecessors as Key and Finance Minister Bill English try to fulfil their effective election promise of a return to surplus by the 2014-15 year.
The pair also have their fingers crossed that the partial share float of Mighty River Power - the first state-owned enterprise up for partial privatisation - will be heavily oversubscribed and thus a big success. However, as National's opponents like to remind Key and English, selling public assets does not amount to a growth strategy. Neither does gutting the public service.
The joking, more lighthearted persona that Key previously exhibited would be dreadfully out of sync with all this.
He may be looking more serious for other reasons. The question haunting him is what kind of legacy he leaves as prime minister. Will historians simply categorise him as someone who was extremely good at managing his Government and winning elections, but whose arch pragmatism produced nothing of lasting substance?
The trouble is that the kind of reform required to elevate him to the ranks of one of the great prime ministers is unlikely to make him popular. One thing poll-driven Key does not like is unpopularity.
It all adds up to a rather bleak picture. If Key is looking wary, it is hardly surprising.
