Key added that "from memory," those had pretty much been the same rules that had applied when Peters had a very similar relationship with Labour between 2005 and 2008.
The Prime Minister could have added that NZ First had been opposed to Labour's negotiation of a free trade agreement with China, but that had not resulted in Peters losing his job as Foreign Minister.
Phil Heatley, on the other hand, might have been happier yesterday had he suffered a bout of amnesia.
The Housing Minister was asked by Labour MP Annette King if he still stood by his month-old statement that Housing New Zealand "clients" were now getting a much better service since the corporation closed branch offices and replaced them with an 0800 number.
This prompted a sheepish admission from Heatley that the transition had "not been without its challenges" and extra staff were being employed to sort things out.
And then there was the strange case of National's Gerry Brownleee, now Transport Minister, and the Greens' Julie Anne Genter, a fresh-faced newcomer with responsibilities for transport policy.
Genter's questions - delivered with the earnestness and gesticulations of a university debater - focus heavily on the supposedly pending apocalypse of oil shortages and spiralling petrol prices which, according to the Greens, make National's expensive road-building programme redundant and wasteful.
Brownlee, who exhibits the demeanour of a Gulliver having his heel pricked by some Lilliputian in the faraway backbenches, promptly develops an instant amnesia for what might happen in the future and waves Genter's dire warnings aside like a doctor saying there is nothing to worry about.
She in turn suffers a similarly fast-acting amnesia to the contents of his reply and presses on with another pre-prepared Armageddon-beckoning question. This talking past each other continues until Gentner mercifully runs out of questions.
Amnesia, however, may well be an unavoidable affliction in politicians. If they were to remember everything they might have believed, said or done in the past, they would be paralysed by indecision in the present.
Amnesia may just be another word for flexibility.