The man told the Herald on Sunday he had apologised to his wife and their children. "This has almost destroyed my marriage. My daughter and son don't really talk to me. They think I am a crim."
He said his wife was "sort of" supporting him, but he said he could not blame her for her feelings towards him. "I know exactly how she is feeling."
Physically it had taken a toll on his health. "I'm on pills for high blood pressure. I could hardly lift my arm at one stage. I put on 10kg in six months," he said.
And fighting the case had also left the family with "huge financial issues", adding "half my life savings" had been spent.
Despite name suppression, many people were aware of his identity and he was struggling to get work.
"I just actually lost out on a job - I was working for this company but there was a piece about this woman's husband screaming abuse at me. One of the shareholders of the company who lives in Dunedin said, 'I can't employ you'," he said.
In recent times he said he had resorted to applying for "menial jobs".
He had also had "rapist" spray-painted on a wall outside his home.
The man blamed the massive life upheaval, for himself and his family, on "20 seconds of madness".
"It was the moment of madness ... I don't know what came over me," he said.
During the past four months, the man has been compared to sexual predators such as Rolf Harris and Jimmy Savile. He said he now wished he had ignored legal advice and pleaded not guilty to the charge of performing an indecent act.
But Hemsley told the Herald on Sunday: "My only comment to [the man's] statement of me being consensual is why did he plead guilty twice? If that was true he would have pleaded not guilty."
In a TVNZ interview last week, she said she remained upset that the man had gained name suppression.
"It's just so wrong. There is no justice in that at all," she said.
Hemsley was also critical that the man had received a discharge without conviction.
"How can you plead guilty and get away with nothing? He's just walked away with his head held high basically."