"He couldn't seem to remember things. He said he had to write them down in a book."
On the day of his death, she said she understood he'd asked the neighbours who were supporting him to wake him later in the day and on going to do so they found him dead.
"I don't think he wanted to go to jail. He was scared ... I just came home and saw the police cars everywhere and then a hearse turned up."
Another neighbour who also didn't want to be identified said it was a sad end to a life.
"He was just an old man facing his own demons," she said.
Police say there are no suspicious circumstances surrounding the death which they are investigating on behalf of the coroner.
Ball, who was facing charges of sodomy, indecent assault and performing indecent acts, last appeared in court in July when he was granted an extension of time until September 17 to plead to the charges.
Wairarapa Rape Crisis counsellor Michelle Ihaka said she was unable to comment on how the death would affect the victims as everyone dealt with things differently.
However, they had come forward seeking justice, which they had now possibly been denied, Ms Ihaka said.
"They did come forward wanting justice. They could be dissatisfied with that outcome.
"They could be dissatisfied at sentencing. It's a no-win situation. They still have to live with what has happened to them," she said.
Detective Sergeant Bill van Woerkom of Masterton police district child-protection team said known victims of the alleged sexual abuse had been advised of the death.