The victim told the Times she hated him living next door: "When I see him I feel very shaky and that's how he affects me, it just brings everything back.''
The man had been described by Judge Kevin Phillips as a high-risk recidivist offender with limited controls and a strong sexual drive.
However, he told the paper that although he knew his victim lived in the house, he had nowhere else to live.
"If I can get over it, why can't she? It's past tense,'' he told the Times.
Police have visited the man and encouraged him to move, but have no legislative powers to force him to relocate.
Rape victim groups yesterday said the law was poorly equipped to give victims of crime the protection they needed.
"It's absolutely unacceptable - something like this shouldn't be allowed to happen,'' said Dr Kim McGregor, executive director of Rape Prevention Education.
Louise Nicholas, advocate for rape survivors and high-profile victim, echoed that opinion and said she felt for "that poor woman''.
"When you've gone through that horrendous crime and you see that person face-to-face even after you've been through the court system, honestly there is nothing that kills you more.''
There are no mandatory programmes for convicted sexual offenders - they must choose to take one while in prison, she said.
When they are released from prison, offenders must pay for themselves if they wish to follow up their rehabilitation.
"There are no funded community programmes to deal with adults who have sexually offended against other adults. We have a lot of sexual offenders who want help but they can't access it at the moment because they can't afford it - it's shocking.''
Ms Nicholas said legislation needed to be passed to stop offenders from ever going near there victims again.
Dr McGregor said people should remember that in New Zealand, only 1 per cent of sexual crimes result in a conviction.
"So that 99 out of a 100 sex offenders are living in our communities. They are already living next door to the victim, in the same house as the victim or in the same community as the victim ... So New Zealand as a whole community is living with this situation every day.'
The Case
2004: Sentenced to four years in prison after admitting raping the woman.
2008: Release conditions stipulated that for six months he was forbidden to go anywhere near his victim's then address.
2010: Jailed for 12 months after continually returning to another woman's house to steal her underwear
2010: Conditions run out and he can go near victim.
2011: The woman discovered he had moved next door to her.
