"I accept that the way the press conference transpired left room for all the opinions and views and criticisms that subsequently flowed," he said. "I accept that it was distressing for the complainant, and I wrote to the complainant a few days later and apologised to her. Holding a press conference with Mr Sutton was a mistake. I accept full responsibility for that mistake, and deeply regret the hurt that arose from it."
State Services Minister Ms Bennett said she was "struggling to see how any part of that Monday [press conference] was done correctly, quite frankly".
"I'm not happy with any part of it but I think we have identified that; he's accepted responsibility for it and I expect some very serious lessons to be learned."
She said the review of the State Services Commission's guidance to public servants on workplace bullying and harassment announced by Mr Rennie last week was "the right thing to be doing to make sure we do have the right policies in place".
She expected "absolutely no repeat" of the errors Mr Rennie had now apologised for and when asked if he would lose his job for further similar mistakes said: "I would not expect anything near this kind of mistake ever again."
Mr Rennie yesterday told media he had thought "long and hard" about resigning, but had decided not to, having discussed the situation with ministers. Ms Bennett said he had not discussed that with her.
Mr Little said it was "good that there's been an acknowledgement that he got it wrong" but there were still questions over Mr Rennie's judgment, particularly when he initially said he would have let Mr Sutton's contract expire if he hadn't resigned rather than sacking him. That "sounds particularly timid in the face of a finding that [Mr Sutton] committed serious misconduct", said Mr Little.
"I still think he should be stood aside pending an investigation into his decision-making, how he made that particular decision if not a number of others, to see whether he is fit to continue in the job."
PSA national secretary Richard Wagstaff said that by holding a joint press conference with Mr Sutton, Mr Rennie gave Mr Sutton an opportunity to put his side of events "that was denied to the complainant".
However, Mr Rennie's acknowledgement of his mistakes was "the first step to making improvements".
"Mr Rennie has acknowledged that his actions could leave public servants with less confidence to come forward in the future. The key now is the way forward to restore any trust and confidence in the SSC's processes that may have been lost."