William, then 15, and Harry, 12, were at Balmoral with their father when Diana died. In candid interviews to mark the 20th anniversary of her death, the brothers revealed their regret that they cut short their final phone call with her, just hours before her death, because they wanted to go and play.
They said their grief was "still raw" but also recalled treasured memories of their mother.
They released photographs from her personal album, including a picture of Diana holding a baby Harry, taken on the royal yacht Britannia by Prince William.
Speaking of the divorce, Harry said: "The two of us were bouncing between the two of them and we never saw our mother enough or we never saw our father enough.
"There was a lot of travelling and a lot of fights on the backseat with my brother, which I would win. I don't pretend we're the only people to have to deal with that. But it was an interesting way of growing up."
William and Harry spent ten days with their mother and Fayed, the son of then-Harrods owner Mohamed Al-Fayed, at his villa in the South of France.
But while they returned to London and then Balmoral, she joined Mr Fayed on his yacht around the Mediterranean, and went to the Greek islands with a friend. She also travelled to Bosnia to campaign against landmines.
Diana remained in frequent phone contact with her sons, but they told how their final conversation lasted just five minutes.
Harry said: "I never enjoyed speaking to my parents on the phone. We spent far too much time on the phone rather than speaking to each other. I can't really remember what I said but I regret how short the phone call was. I'll have to deal with that for the rest of my life. Not knowing that was the last time I was going to speak to my mum, how differently that conversation would have panned out if I'd had even the slightest inkling."
William said the call interrupted a game with their cousins Zara and Peter Phillips, and said he and Harry were in a "desperate rush to say goodbye" so they could go back to playing.
He said: "If I'd known what was going to happen I wouldn't have been so blasé about it. But that phone call sticks in my mind, quite heavily."
William said her death was "utterly devastating" and he and Harry struggled to understand their feelings or speak to each other about their grief. They praised Diana as "the best mother ever" who left them with a personal legacy of her love, as well as her global charitable impact.
PRINCES BARE THEIR SOULS: IN THEIR OWN WORDS
Harry on Diana's hugs
"She'd engulf you and squeeze you as tight as possible. I miss that feeling."
William on her loving nature
"She was extremely good at showing what we meant to her, how important it was to feel."
William on his wedding day
"Not many days go by that I don't think of her. I did really feel she was there ... I very much felt she was there for me."
Harry on her sense of fun
"She was a total kid through and through. I can hear her crazy laugh in my head."
... and her sense of mischief
"One of her mottos was 'be as naughty as you want, just don't get caught'. She was one of the naughtiest parents. She would smuggle sweets in our socks."