"The fascination with the game is the intimidation part of the game and the violence that we have on the defensive side.
"If you don't have people worried about getting hit, or getting hit hard, (then) anybody can go out there and catch the ball, anybody can run the ball.
"The game has been limited, because part of the game is intimidation, part of the game is violence, and that (focus on safety) changes the game.
"They've been super cautious on that aspect of it but you have to have contact to learn how to tackle correctly. And by eliminating contact in the college levels and high school levels hurts the game from my perspective."
Listen: Dr Ann McKee, expert in neurodegenerative disease, talks to Kent Johns about CTE and concussions in sports
While concussion remains an issue in all oval ball codes, Ellison believes rugby is showing the way forward with its emphasis on teaching young players correct tackling techniques.
Rather than wrapping young players in cotton wool, the NFL would become safer if young players were given a better grounding in proper defensive techniques.
He fears the end result of all the precautions will see the game lacking in all the areas that made it so appealing to players and fans.
"The big discrepancy I've seen is we've gotten away with fundamental teaching of how to tackle," he said.
"Where I think rugby is much better - but rugby still has the same concussion level, let's not hide that - because it's a hard surface you're going to get concussed.
"But if you're taught how to tackle correctly you are going to reduce some of that aspect of it.
"But I love the violence of it, or else we're going to be playing touch football or touch rugby. And touch rugby is not rugby, right? It's the same thing."
Having suffered numerous concussions has not changed his views on the game he loves and Ellison believes genetics play a big part in the varying degrees in which players are affected by head knocks.
"I think it's genetic. I've done some work with the military, because an IED (Improvised Explosive Device) blast in Afghanistan, inside that circle you have some guys that are concussed and some guys aren't.
"And just like a boxer with a glass jaw, some guys can take hits. If you can't take it, get out of the game.
"So there are people playing the game that shouldn't be playing the game."