Now 62, and with a tabloid sting behind him, Todd insists his days of wild partying are over. "I don't know," he says. "Maybe the young still do it. But I can't perform the next day if I've had too hard a night.
But it got pretty wild back in the day? "Well, it was all much more amateur back then. I think most sports have got way more professional haven't they? And we have, too. I mean, we used to perform with hangovers most of the time. But now, it's just way more professional. And if you want to be competitive it does affect your judgment."
We are up at Burghley Horse Trials, in the bucolic surrounds of the magnificent Burghley House, designed by William Cecil, Lord High Treasurer to Queen Elizabeth I, and Todd is driving me around the cross-country course in a Land Rover provided by his sponsors.
A seven-time Olympian and twice an Olympic gold medallist, Todd will this weekend be attempting to win his sixth Burghley title, 38 years after he went for his first. Riding Leonidas II, he takes part in the dressage on Friday, the cross-country on Saturday and finally, on Sunday, live on the BBC, the show-jumping.
It is a gruelling schedule for a rider now in his seventh decade, but Todd does not seem daunted in the slightest. Lean and agile, Todd stops the car every now and then to pick his way through the barriers keeping out the spectators - all of whom nudge each other in the presence of eventing royalty - to get a closer look at some of the jumps and fences.
"They call this one Storm Doris because the logs are from trees which came down in the storm," he explains of one zig-zag-shaped fence.
Pacing out the distance from one wooden corner to the next, Todd explains what the course designers are trying to do.
"You can't take these 'corner' jumps too tight because the horse might just run sideways around it and you get 20 penalties," he says. "But take it too deep and... it's a f------ long way from here to here." Todd - who rates himself "third or fourth favourite this weekend" - shows no sign of wanting to hang up his spurs. Leonidas II is 13 - "in his prime" - and you're only as old as the horse you're riding. "And I've actually got a really good young horse, too."
Tokyo 2020 remains a possibility. "I definitely want to go to the world championships next year in America," he says. "And then we'll see. It's only two years more after that, but two years when you're 62 or 63 ... watch this space."
Don't write him off. Todd tried retiring once, after winning bronze at Sydney 2000, returning to Waikato with his wife, Carolyn, and their two children to breed horses. But he found it overrated, eventually returning to the UK in 2008 to continue competing.
"I never thought I'd come back full time," he says. "It was just a bit of a dare to see if I could make it back to the Olympics in six months."
He could. But it cost him his marriage. He and Carolyn separated in 2009, although they remarried in 2014 and now live together again in Wiltshire. "She is coming up later," he says as we arrive back at his motor home in the competitors' paddock where Zara Phillips - also competing this weekend on High Kingdom - and her husband Mike Tindall stop for a brief chat.
Later, they will all head off to Burghley House for a cocktail party.
Todd, having won best dressed man at Wednesday's "trot-up", where the competitors all parade their horses (and themselves), will no doubt be in hot demand. But he says he will just be looking forward to getting back to his motor home. "We have friends coming over to our lorry for dinner," he explains. "And I need an early night."
