The decision was prompted by the police decision on March 26 not to lay charges against Ambrose. At that time, Assistant Police Commissioner Malcolm Burgess said investigators had concluded the recording was more likely than not to have been deliberate.
Fulstow said the move was also prompted by "the public interest in bringing the matter to a close".
Ambrose said he was "relieved", but indicated he could not have paid anyway.
A spokeswoman for Key said:
"It's a decision for the Solicitor-General and any questions should be directed to him."
The application for costs, and their withdrawal, was made in the Attorney-General's name, but day-to-day litigation decisions are made by the Solicitor-General.
Political commentator Bryce Edwards said the decision was "a pragmatic political decision to back down".
"It's not one made on a matter of principle or logic."
It was ironic hearing the news on May 3, World Press Freedom Day, Edwards said.
"It was an issue of press freedom and the ability of journalists to go about their work. The initial decision would have impinged on that."
Ambrose said his lawyers were still deciding whether to take defamation action against Key for saying Ambrose broke the law. He is also still waiting for police to return $1000 worth of recording gear, despite repeated requests.
Auckland police spokeswoman Noreen Hegarty said police expected to contact Ambrose last week but, as of Friday afternoon, he had not heard from them.
Ambrose said he lost "tens of thousands" of dollars as work dried up during the scandal.
He was "pretty much back on track" with work but he would probably have moved to Australia if not for his children.
"I became completely disillusioned with the people running the country. And that's coming from someone who's been a National voter for 18 years."
cherie.howie@hos.co.nz
