Letterman saved the most hot-button question for the end of the interview, pausing before saying, "You and I have one other thing in common, and I have to be cautious how I articulate this."
"A few years ago, I got myself into some trouble, and the situation was my responsibility and my fault," Letterman continued. "I did something that I had no business doing, and I regret it, and since then I have tried to acknowledge that mistake and be a better person."
Facing blackmail threats in 2009, the married late-night host admitted to cheating on his wife, Regina Lasko, by having sex with female members of his staff.
Jay-Z also became headline fodder for cheating on his wife, Beyoncé. Speculation about the rapper's infidelity began in 2014 when a video surfaced of his sister-in-law Solange Knowles attacking him in an elevator. Those rumours gained steam with Beyoncé's 2016 record "Lemonade," which contained several semi-veiled references to cheating, such as "You can taste the dishonesty/It's on your breath as you pass it off so cavalier" and a mention of the mysterious "Becky with the good hair." Jay-Z finally admitted his transgression last year in a November interview with the New York Times's T Magazine.
"At the time, the pain that I caused myself was the fear that I had blown up my family," Letterman told the rapper. "I never talked to a person who had been in that situation. And I'm wondering if this rings a bell with you."
Indeed, it did. Jay-Z said he didn't have the emotional tools required for a marriage, but through therapy and Beyoncé's support, he's working toward gaining them.
"For a lot of us, especially where I grew up and men in general, we don't have emotional cues from when we were young. Our emotional cue is 'Be a man. Don't cry,'" he said, mentioning his song "Song Cry," in which he raps: "I can't see 'em coming down my eyes, so I gotta make the song cry."
"It was my way of saying I want to cry, I want to be open, I want to have the emotional tools that it takes to keep my family together," Jay-Z said. "Much like you, I have a beautiful wife who was understanding and who knew I'm not the worst of what I've done."
"We did the hard work of going to therapy," Jay-Z added. "We love each other, so we really put in the work for years."
Because of that, Jay-Z said: "I like to believe we're in a better place today. I'm proud of the father and the husband that I am today."
Jay-Z said he shares his story in case someone watching who has lost hope in a relationship relates to it and thinks, "Let me stick with this and come through the other side. I see how that looks."
"We didn't see that. All we saw is people fleeing," he told Letterman. "We just see people give up. The divorce rate is 50 percent. We never see people (who say) 'OK, let's work through this. I love you; I love my family.' "
On his mother, Gloria Carter, coming out as a lesbian
One touching song on Jay-Z's 2017 album "4:44" is "Smile," which reveals that his mother Gloria Carter is gay, with the lines "Mama had four kids, but she's a lesbian/Had to pretend so long that she's a thespian." Carter herself appears at the end of the song with a spoken word poem, saying: "Life is short, and it's time to be free/Love who you love, because life isn't guaranteed."
The rapper told Letterman that he has long known that his mother is a lesbian, but it wasn't until he was recording that album that the two spoke about it. That conversation was the first time he heard his mother say she loved a romantic partner.
"The only thing about that experience for me is her," he said. "Imagine having to live your life as someone else, and you think you're protecting your kids."
He said the conversation was so emotional that he began crying. The next day, he sat down and wrote "Smile."
"For her to sit in front of me and tell me, 'I think I love someone.' I mean, I really cried," he said. "I cried because I was so happy for her, that she was free."
On making peace with Adnis Reeves, his absentee father
Jay-Z's lyrics often center on his father, Adnis Reeves, who abandoned Carter, Jay-Z and his three siblings when the musician was 11 years old - leaving behind a deep emotional scar. When Beyoncé was pregnant with the couple's first child, Blue Ivy, Jay-Z rapped a mission statement: "My dad left me and I promise never repeat him."
But time, Jay-Z said, helped him better understand his father and the circumstances leading to his absence. Reeves' brother was killed in the projects, and finding the murderer became something of an obsession for him.
"Someone would call and say, 'I just seen the guy who killed your brother,' " Jay-Z said. "He would get up from his bed - his bed with his children - and he would take his gun, and he'd leave the house."
Eventually, Jay-Z's mother confronted Reeves, saying " 'Hey, you have a family here.'"
"She didn't have the language that she needed to speak to him, like 'We love you. We don't want to lose you, as well,' " Jay-Z said. "Her fear came out almost like an ultimatum to him. ... That kinda like splintered their relationship. From there, he was in deep pain, started using heroin, and things like that."
As a young man, Jay-Z felt abandoned by and furious with his father. But now, he said, "I have compassion for what (my dad) went through, which was anger."