Mr Dunne resigned as a minister on Friday, insisting he had not leaked a report on the GCSB spy agency to Fairfax reporter Andrea Vance.
Mr Key required Mr Dunne's resignation for not co-operating with the inquiry. "That speaks for itself," Mr Key said yesterday.
Hard copies of the report were distributed to 34 people before it was released.
Inquiry head David Henry focused only on the three people of the 34 who had been known to have contact with the reporter over a two-week period. Mr Dunne refused to hand over 86 emails between him and Miss Vance but showed Mr Henry a version prepared by him relating to 41 of the 44 emails he sent to her, of which 14 had been partly deleted and four had been completely deleted.
Mr Peters resisted taunts yesterday by Mr Key to release emails between the politician and the reporter, which over the weekend he claimed to have.
Mr Peters toned down suggestions he had a lot of evidence.
"I have sufficient to have said on day one with the greatest of confidence, 'Mr Dunne, you're the leaker'," Mr Peters told reporters.
"If you haven't got the complete picture, which I admit I haven't got, but I've got enough to say what I've said and know we're going to make it home. Otherwise this car has gone a long a way on no petrol."
Mr Peters repeated his assertion in Parliament that Mr Dunne was responsible for more than one leak and he called on Mr Key to use his powers to get electronic records of Mr Dunne to show them. "We want to see the complete package."
He cited other stories he believed had been leaked by Mr Dunne: low morale at the GCSB; the new head of the GCSB; a briefing about the GCSB; and a story about the intelligence and security committee.
The Prime Minister was responsible for security and for his minister and the fact that the GCSB report leak was not an isolated incident "demonstrates how incompetent this Prime Minister is, and how irresponsible his behaviour is".
Labour wants a privileges committee inquiry to convene because of Mr Dunne's denials but in the hope that electronic records could be subpoenaed. Speaker David Carter is yet to rule.
Mr Key admitted in Parliament he still did not know whether Mr Dunne had leaked the report. But the only way he could get the emails would be to reconstitute the inquiry and give it the powers to subpoena the emails.