"I had to walk out halfway through, smoke a joint, just to steel myself for the rest of the thing," the War for the Planet of the Apes star said.
Harrelson appeared on the show to promote his new film LBJ, in which he plays former president Lyndon B. Johnson.
Describing LBJ, Harrelson said "he wasn't publicly vulgar like our president incumbent".
According to the Daily Mail, Harrelson was pictured with Ventura and Trump on Christmas Eve 1998 after having dinner. It's not clear how many times Harrelson has dined with Trump.
It comes after Harrelson revealed how his wife, Laura Louie, found out about his infamous 2002 foursome.
"I'm not sure how Laura found out, but she did," he told the Hollywood Reporter.
"I was kind of hoping she wouldn't see it. I can't remember the details; I've doubtless repressed it. She never saw the thing. But someone told her."
As Harrelson recalled, he was approached by two women after a night out at a London hot spot 15 years ago. The trio was joined by another female, who hopped into a car with them as they headed back to his place. "These girls came up to me, these two girls, and asked me, 'So you want to take a walk on the wild side?' So I said, 'I guess I do,'" Harrelson explained.
The actor revealed the third female passenger was a member of the press, who also enlisted a photographer, after manipulating the other two women. "I was bummed when they all left together 'cause I knew it was going to happen [an article]. News of the World, it was. I never read it 'cause I didn't want to read it," he shared.
When it came time to speak with his other half, Harrelson wasn't expecting her response.
"Laura - this really gives you a sense of the depth of her compassion - what she said to me after finding out was, 'That must be really hard for you, to have this s**t exposed,'" he said.
"She just said that. Now that doesn't mean she wasn't upset."