Byrne went on to say that Spacey displayed an "element of absolute abuse of power" that he also saw in Harvey Weinstein.
Spacey's sexual misconduct also wrecked havoc on at least three other productions, with his television series House of Cards forced into a hiatus after he was accused of sexual misconduct.
It was announced on Monday that filming would resume next year on the Netflix show with Robin Wright Penn staying on in the lead role.
Netflix will not however be releasing Gore, the biopic about Gore Vidal that was filmed over the summer starring Spacey as the famed author.
Meanwhile, reshoots just wrapped on All The Money in the World after director Ridley Scott made the decision to cut Spacey from the finished film.
Byrne also spoke about the three movies he filmed with disgraced producer Weinstein, which included the 1991 film Into the West on which he met his future wife Ellen Barkin.
That film is also now marred for Byrne, with Weinstein's then-assistant Laura Madden detailing the sexual harassment she endured at the hands of her boss during their stay in a Dublin hotel near the set of the movie.
"I knew he was a vile bully and I saw his bullying up close," said Byrne.
"I saw him be absolutely appalling, not just to women but to men as well. He had very little respect for any kind of human being."
Byrne also said that he heard rumblings about Weinstein over the years, but never anything concrete.
"I heard that once or twice from two very well known actresses, but the problem when you hear something like that is, do you pass that on?" noted Byrne.
"Because if it's not true it's awful, and if it is true it's not your job to say, 'Well, I wasn't there, the door was locked, I don't know what happened, I just heard the story'."
He followed that up by stating: "But I did not know, and many people didn't know, the extent of the violence that he perpetrated on women."