Although, around this time, Fleetwood's marriage to his first wife Jenny - they have been married and divorced twice - was also on the rocks and, just to add to the drama, on the world tour for Rumours he and Nicks started having an affair while in New Zealand.
"It was unbelievable that all five members had broken up at the same time with their significant others and that's how we went in to make that album. So it's a very memorable period in all our lives," he remembers with a laugh.
So if you combine the emotional turmoil with the amount of drug and alcohol excess that was going on - which they could now afford with the windfall provided by Fleetwood Mac's four-million-or-so sales - it sounded like it must have been a wild and turbulent time.
Not so, says Fleetwood.
"It was obviously an incredibly difficult album to make with everyone in the band desperately unhappy, but not in a nasty way - just emotionally, with all the break-ups of the marriages and the relationships."
The 65-year-old has told the "well-worn story" of Rumours many times, yet he's keen to explain his reasoning behind why, given all the obstacles, it turned out so damn well. Because as pop-rock albums go, they don't get much more famous than Rumours, and thanks to songs such as Buckingham's Go Your Own Way and the oddly seductive sadness and dancey lope of Nicks' Dreams, it is one of the top 10 best-selling albums of all time, with 40 million copies sold.
"The great part of it is that we were so dedicated to our musical focus, having come off making Fleetwood Mac, which, for us, did really, really well. We never had anything vaguely like that happen to us, even back in the day in England. We knew that we had a very exciting musical opportunity to keep going forward, so we did it against seemingly hopeless odds, and without strangling each other," he laughs.
"But it was definitely not an unpleasant and nasty thing, it was just incredibly thought-provoking and incredibly sad. We were all very vulnerable and I think having been in that position, the album became a chronicle of that dynamic that I've been trying to describe, which was incredible emotive sadness. A sense of, 'Could these people get back together?'
"So all we had was the music that we played, and that was how everyone communicated."
If your old vinyl copy is a little worse for wear, the 35th anniversary reissue was released in New Zealand this week - and the super deluxe edition is quite a package, with five discs, including the remastered album, a 1977 live set, two discs of rarities, a DVD of The Rosebud Film about the making of the record, and a vinyl copy of the album. It also comes in a three-disc deluxe edition and a standard release of just the remastered album.
Admittedly, the band's more accessible direction was ushered in on 1975's Fleetwood Mac, the first album to feature the poppier songwriting bent of Americans Buckingham and Nicks with the latter's Rhiannon a brooding and gutsy highlight.
But for Fleetwood it was Rumours that truly took the group from hard-rockin' British blues band to mainstream pop-rock mega stars - and, 35 years on, he reckons it stands up well.
"I really like the album," he says simply. "And the other part of it, which was a blessing, is when we crafted that album, as artists, players and musicians and the sensibilities that we had, we didn't get caught up in the latest gizmo stylings of recording."
Because rather than relying on the musical gimmicks many artists were using in the late 70s, such as drum machines and "obscene sound effects and certain types of echo that really dates music", Fleetwood Mac kept it real and pure.
"We produced it ourselves and we didn't have some whizz-kid producer who came in and said, 'Well, this is the sound that's hitting the radio these days ...' We so weren't in that world, thankfully, and I think you really can't tell when Rumours was made and it makes that album sonically honest. It's just a straight-ahead, really well-recorded album."
Fleetwood says the change to a poppier sound for Fleetwood Mac and Rumours was not that dramatic, because the band had already been through a number of different incarnations since starting out in London in 1967 when he and original guitarist Peter Green teamed up. There was the initial "straight-ahead blues" period, and then when Green left in the early 70s, he was replaced by Bob Welch who helped the band move in a more melodic direction. So Buckingham and Nicks joining was a natural and welcome progression.
"We weren't really aware of making that transition, we drifted into it in a very organic, normal way. Certainly, someone who picked up a Fleetwood Mac album five years before might have thought, 'Oh my God, what happened to them? This is all harmonies and melodies and not the blood-and-guts band I remember.' I'm sure that did happen to certain listeners.
"But for us it was really quite normal and gradual."
With his long, lean, 2m-plus frame and his ever-present beard, Fleetwood is arguably the most recognisable of the band, alongside Nicks and her big blonde hairdo. He's also been the cover star on many of the band's albums, starting with 1968 albums Mr Wonderful and English Rose, for which he was dressed in drag.
"Those were the start of using the creature known as Mick as an icon thing," he says.
On the Rumours cover he is posing, seemingly, rather romantically with Nicks - until you spy a pair of wooden balls ("my balls", he calls them) hanging strategically below his crotch.
The balls were a gimmick he has used right from the early days of the band and to this day he never walks on stage without them hanging off his trousers.
"I was never really a very clever drummer, so I would get up from the drums and do a ball solo. I'd grab my balls and start whacking them on the microphone. And one thing led to another," he laughs.
To be honest, Fleetwood - a self-confessed "nut" - got up to all sorts of other shenanigans on stage and off, which included flashing and showing off various parts of his anatomy... but then that's a whole other story.
Who: Fleetwood Mac drummer Mick Fleetwood
What: The band's 1977 album Rumours reissued 35 years on
On tour: Fleetwood says the band is looking to return to New Zealand at the end of the year or early 2014.
- TimeOut