All Bill Spedding had done was go and fix a washing machine at William Tyrrell's foster grandmother's home.
But after being wrongly accused of the 3-year-old's abduction, he says it's damage that can never be repaired, news.com.au reported.
"Describing the feeling is very difficult, but it's sad," Spedding told Four Corners.
"We just don't know, when someone's going to pop out of the woodwork, and grab me around the throat again, and say, 'You're Bill Spedding. What'd you do with William?'"
Spedding became the highest-profile person of interest in the police investigation into the disappearance of William, who vanished from outside his foster grandmother's home in the NSW mid-north coast town of Kendall on September 12, 2014.
The washing machine repairman had been there three days earlier.
Spedding has accused police of wrongly targeting him.
He told Four Corners that investigator Gary Jubelin, who took over the case from February 2015 after Spedding had been identified as a person of interest, said he was coming after him.
"Gary Jubelin said to me 'Mr Nice-Washing-Machine-Man, we're going to ruin you'," he said.
"And at another point of time he [said], 'Whenever they talk about William Tyrrell, your name's going to come up as well. And when they're talking about you, William Tyrrell's name is going to come up'."
Jubelin said he denied saying those words and that it was the right decision to keep investigating Spedding.
He become the main person of interest after detectives raided his home and business in January 2015.
Spedding and his wife Margaret were casually sitting on their back veranda having a coffee and feeding magpies and kookaburras when police cars swarmed their house.
The search was sparked by a tip-off related to allegations made against him 30 years earlier that he had abused two children. The allegations has been investigated and dismissed at the time.
In an unfortunate coincidence police found a Spider-Man toy in the back of Spedding's van. William was wearing a Spider-Man suit when he disappeared.
Spedding said after a six-hour interview at the police station, one of the detectives gave him a warning.
"We know you did it. We're going to get you. I'm going to come and arrest you," Spedding said he was told.
"I said, 'I haven't done anything. What are you talking about?' And that's when I realised that they really thought that I was involved in some way."
"If a fella was guilty, I could well see why someone would break. It was intense."
Spedding said on the day William disappeared he went to met his wife for a coffee before going to the Laurieton Public School assembly to see their grandson get an award.
When he tried to tell police he had evidence on his bank statement of purchasing the coffee he said he was told by an investigator, "I'm tired of your bullshit and lies".
But the "knockout blow" came when police blindsided Spedding, arriving at his property in April 2015 to arrest him over the historic allegations that dated back to 1987.