He asked about Judith Collin's comments National would be "a bunch of wusses" if it raised the Youth Court age to include 17-year-olds.
Key deliberately misheard this as 70-year-olds and assured Peters he was safe.
Peters followed up by asking how Key could have confidence in his Police Minister, Health Minister and Finance Minister since they did not seem to have confidence in each other.
As Key insisted National was a den of harmony, despite appearances, Peters ramped up his question line.
He pointed to "spills coming out of caucus" about "terrible instability, feuding, backstabbing, fighting, all sorts of secret calls - so much so that it has fallen to New Zealand First to look like the epitome of stability?"
This did prompt a moment of harmony from National - a unified chorus of laughter and Key retorted that it was easy to have stability in what was effectively a caucus of one.
Green Party co-leader James Shaw also decided to have some fun on the PM's last day, given the PM's endorsement of English as his replacement.
He asked Key first about Coleman, asking if Key even knew what he looked like. Then he moved on to Collins. He asked if Key was confident that "if Judith Collins becomes Prime Minister, New Zealand will not wake up one day and find itself tied with Zimbabwe on the Transparency International corruption index?"
He then moved on to English. "Is the real reason New Zealand's productivity is so low because every working-age New Zealander has been bored to death listening to Bill English?"
Strangely, even English nodded his head at that one. But Key defended him. "If that is his test, then I should introduce him to his own caucus colleagues. Man, they are not exactly people I want to party with when I leave Parliament."
Shaw wound up by asking whether Key wanted to change his mind now he had seen those in line to replace him.
At this point, Shaw was ahead on laughs, but Gerry Brownlee stepped in to help out the PM. He wanted to know if Key could ever recall a day when the Greens had put so much effort into their questions.
Key's response was a dig at the Greens' likelihood of remaining in Opposition in perpetuity: "No, but it is good to see that they are getting the hang of it, because they are going to be asking questions for a very long time."
With that, Key's final word in Question Time was done.
While Key ticked off his lasts, Key's caucus were left to grapple with a first for many of them: the leadership change.
National's rules provide for a caucus vote to elect a new leader, which means this is a fairly brief period of existential angst compared to Labour's.
But Labour's Chris Hipkins cheerily warned them the end was not in sight.
"I've read this book before, and I can tell you now: it all gets worse from here. Whoever is the leader, the plotting will continue."