"Remember when we brought Paul Henry back to Breakfast? I'd already hired Paul twice before. He always leaves. He gets bored, he has a lot of money so he always leaves," he said, adding that Henry is "a good guy to work with".
"He came on and he did great on ratings in the morning, he really put it on the map."
Harris said that, while everyone was celebrating the ratings Henry was bringing in the morning, he could tell the overall results were "not great".
"The overall rating was sliding.
"We weren't even rating overall despite the fact that he was killing it with Breakfast.
"I said at the time this wasn't great. He's gonna leave and when he leaves we're gonna be in trouble," Harris said.
He says he was seen as "raining on the parade" but believes he was just being "realistic".
"They didn't listen to me and they should have. I really know this business."
Harris blames the results on "a bunch of people from television who knew nothing about radio" making bad decisions.
"They didn't want to listen. So that's why it's frustrating for me."
The radio show host went on to say that, despite media reports, "it's not about left and right" and the real issue isn't political.
"Talk radio is about ordinary working class people that are probably more conservative than what television people think they are."
According to Harris, the older, 50+, radio audience does not want to hear about issues such as the problems faced by the transgender community. "It's not hate, they're just not interested," he says.
"We're not talking to the people that actually listen to talk radio. Radio is about relatability. That's where we failed."