The Stones fired him and Keys found himself owing money to the band. A good story, but is it true?
"Yeah," he laughs with a weary sigh. "That's what they tell me. It was reflected in my pay cheque so I guess it's true."
Stories about Keys - the Stones' touring saxophonist and who has played on more than half a dozen classic albums with them - are legendary and legion. He was there when the Stones recorded Exile on Main Street in the south of France: "I've seen some of the outtakes of the videos and I swear I do not remember there being a cameraman anywhere, but obviously there was."
He was famously part of the coke and booze days in Los Angeles with John Lennon, Ringo Starr, Phil Spector and others in the early 70s, has played on albums by everyone from Eric Clapton and Harry Nilsson to Marvin Gaye and Lynyrd Skynyrd, and recently wrote his autobiography.
Its title? Every Night is Saturday Night.
He's been rich and he's been broke: "But every musician I've known has been in debt at some time or other. It's just a natural state at times. I made tonnes of money, I just didn't keep any of it. There was no shortage of it, holding on to it was the trick."
Born in Texas, Keys turned professional at 15 when he toured with Buddy Knox, who had a huge hit with Party Doll. He knew Buddy Holly's backing band, The Crickets.
"I knew Buddy briefly but we didn't hang out together. He was the first guy I heard who played electric guitar and sang, but Buddy and I weren't what you would call tight. I was very close friends with drummer Jerry Allison, and Joe Mauldin, the bass player."
In the 60s, Keys toured constantly and by the end of the decade had fallen into the orbit of British musicians, notably the Rolling Stones, whom he'd first met in 1964.
He became their first call sax player (until Jagger fired him after the bathtub incident) and that's him on songs such as Brown Sugar. He laughs about that crazy period between 1969 and 1976 when he was touring with Delaney and Bonnie, and then Joe Cocker, and playing sessions with the Stones, Lennon, Clapton, Gram Parsons and others. And feeling bulletproof.
"There was not a lot of thought given to tomorrow and the early 70s were pretty much driven by rock and roll and booze. I remember quite a bit about those John Lennon sessions. There's a lot of stuff written about that so-called Lost Weekend of his [which lasted 18 months] but that was typical of most weekends. And most week days.
"But there was some great music made. It wasn't all just drinking and carrying on. Great music came out of that period. That stuff with John and Harry and Ringo. It was a good time... but there were a lot of non-musical activities."
Keys was also there when, in 1974, Paul McCartney dropped by the studio where Lennon was recording. It was the first time in years they'd been in a studio together, and it would be their last.
A bootleg of the recordings is called A Toot and a Snore. Keys doesn't remember much about it.
"I was aware of being in the studio with John and Paul. But back then a lot of people got together in the studio. Hell, Ringo was always hanging out, guys like Jack Bruce and so on. John and Paul hadn't been together in the studio for some time but it was fine and everybody got on. There was no blood let."
Despite being fired by the Stones - and he'd been Jagger's best man at his wedding to Bianca in 1971 - Keys came back into the touring band in the early 80s and has been with them ever since.
His has been a remarkable career - he played on Lennon's Walls and Bridges and Rock'n' Roll albums, that's him on the Lennon single Whatever Gets You Through the Night, and has credits on albums by George Harrison, Donovan and Barbra Streisand.
And when the Stones come to New Zealand he is bringing his own band down, a semi-supergroup with players from Georgia Satellites (guitarist Dan Baird), Lynyrd Skynyrd (former bassist Robert Kearns), the Black Crowes (drummer Steve Gorman) and others.
The night before the Stones show they, billed as the Suffering Bastards, will play a Powerstation gig.
"We got together in Nashville to play some afternoons in a friend's place, he has a night club. Just to play some rock and roll.
"Most of the material is something I've had a real connection to, like I've played on the record or known the artists.
"So we do some Stones and Lennon stuff, some Joe Cocker. And that's rock and roll."
And does this rare survivor of the punishing lifestyle still practise regularly?
"Nah," he laughs. "Now I practise getting out of bed."
Who: Bobby Keys and the Suffering Bastards
Where: Powerstation, new date Saturday March 29.
- TimeOut