"I'm not going to rule that out but I'm also not going to rule that in."
His Government's amalgamation of the Ministry of Agriculture and Forestry with the Ministry of Fisheries and the Food Safety Authority into the new Ministry of Primary Industries had been "very logical".
"There's one other particular area we want to focus on in terms of where we think gains can be made and synergies can be found and I'll be able to talk more about that on Thursday."
Mr Key hinted at a fresh emphasis on accelerating economic growth with his appointment of Mr Joyce as a "super minister" across Economic Development, Science and Innovation, Tertiary Education, and Skills and Employment after the last election.
The National Government's push for up to $1 billion in savings has seen the headcount in the public service drop by about 2500 in the past three years, which was "not dramatic" Mr Key said.
"It's my expectation it will probably go a bit lower."
However, proposals to restructure Mfat, which could result in up to 305 jobs lost including diplomatic and policy roles, have seen a stream of criticism from senior diplomats leaked to the media and public.
Mr Key acknowledged there had been some "push back" against the proposals at Mfat. But the final shape of the changes at Mfat was up to chief executive John Allen.
"I think he will take on board the feedback and I'd be amazed if he ends up delivering on all of the things he's consulting on. There will be some change but that's a matter for him."
Some proposals "may be a little bit too aggressive and he might have to step back a wee bit".
Labour's state services spokesman, Chris Hipkins, said speculation about potential amalgamation involving the Department of Labour was rife and had been fuelled partly by the recent suspension of the recruitment process for a new chief executive for the department. The speculation has included suggestions the department's functions could be split up and farmed off to the Ministry of Economic Development and Department of Internal Affairs.
But Mr Hipkins said a major shake-up of the Ministry of Science and Innovation would be "crazy" given it had been restructured only recently.
"You can't reorganise the entire public sector every time you reshuffle the ministerial portfolios."
Meanwhile, Mr Key said his speech in Auckland, which was only yesterday confirmed for Thursday, was not timed to coincide with Mr Shearer's address and had been booked a month ago. Mr Shearer was yesterday giving little away about his speech, saying it would "map out a view of where we're going to go".