Haley's ex-wife said she was shocked to receive an invoice from Harris' accountant for $18,300 of back rent, covering the year her ex-husband lived in Harris' home.
After Haley was sentenced in October last year, she complained to bosses at Manawatu Prison, who launched an internal inquiry.
They failed to take her complaint seriously, she said. "They just seemed to disregard everything.
"The prison said they had dealt with it but they never said how they dealt with it and everything just carried on as normal."
She then wrote to Tolley, complaining about the prison's handling of her complaint.
Harris was placed on leave while an investigation was carried out and he left last week.
In a statement, Tolley said: "The lady involved in this distressing case raised the issue with me and I found it deeply concerning.
"As minister, I cannot get involved in employment issues involving the department, but I did seek assurances from them that a thorough and timely investigation would be carried out.
"Corrections demands high standards from its staff and anyone who does not live up to this is letting down the thousands of dedicated Corrections workers across the country," she said.
Paul Tomlinson, from the Corrections Department, said the original complaint and the subsequent one were acted on as a result of Tolley's involvement.
"The first was dealt with internally, the second triggered an employment investigation.
"As a result, this officer no longer works for the department."
Corrections said there was a "conflict of interest" policy around staff associating with prisoners.
Contacted this week, Harris would not explain why he tried to charge Haley's ex-wife $18,300. "That wasn't motivated by me," he said.
Haley was the subject of an anonymous pamphlet drop around the suburb of Kelvin Grove, outing him as a "confessed paedophile", while he was living with Harris, according to a report in the Manawatu Standard.
"He should not be allowed to stay anonymous in our community, especially among our young children," the pamphlet said.
At the sentencing, Haley's victim read a statement stating her childhood had been turned into a "disgusting nightmare".
The offending began when she was 3 years old in 2000 and continued until she was 9 or 10.