Fair enough. But try explaining this. This Waikato wonder still runs ultra-gruelling ultra marathons including the 60km division of today's Tarawera off-road event, where he lines up with 850 fellow competitors from 28 countries.
Kerry Suter, winner of the first two Tarawera races, says Smith is already top in one department.
"Brian Smith will be the toughest bastard out there," says Suter.
"I hurt every day because of my sport. To maintain this into my 80s seems incomprehensible."
Smith was hurting alright, heading into his 50s. He was overweight, and suffering depression.
His family - wife Betty and the four kids - reckoned a photograph of Brian standing on a giant Moeraki boulder looked like two boulders. A doctor prescribed anti-depressants but Brian headed for the hills. Literally.
"I decided no, I'll find a better way," he says. "You realise something has got to change. Running got me back into life."
What started as a walk-one-run-one-lamppost venture took off in ways barely imaginable for a dedicated non-sportsman whose competitive career encompassed a couple of teenage rugby games.
"I was so bad ... they put me on the wing and made sure I never got the ball," he says. Smith had a good sidestep though. During eight years in the Air Force, he expertly avoided the sports days.
Running pushed a button, initially at the age of 55 while tramping the 78km Heaphy Track. Feeling sprightly on a spare day, he chucked a chocolate bar in a bag and ran the length, finishing in under 12 hours and experiencing his first runner's high.
He doesn't keep a count, but has run about 50 marathons. A favourite memory is being in a Hamilton team, average age 59, which won the over-40s division of an endurance relay in America.
Ultra marathons are a whole new ball game although Smith claims they are easier than road events because his feet enjoy the constant changes of angle. What fun. Smith, who has always trained alone and has no support crew, will spend about 11 hours battling tough terrain between Rotorua and Kawerau today.
Last year, he "tore something at the bottom of the rib cage" at 40km but continued on a serious lean, stopping every few hundred metres to stretch. He has persevered with training for today's race despite a carcinoma being cut out near an ankle and being laid low when a Pitbull-cross "started snacking on my leg".
Last week, he notched 100 training kilometres by running every day.
Watching Smith walk in a measured way around his house, you wonder how a man of this age can even contemplate challenges such as the notorious Tongariro Goat alpine run, let alone finish them.
Finishing is the object these days. Dreams of hurtling towards the finish line are long gone, and the sight of injured competitors encourages Smith to slow down further. Then again, he is quietly proud of leaving about 10 in his wake at his first Tarawera Ultra in his late 70s.
Never stop learning and hope for good genes might be his mantra. Smith makes his own chia seed gel for hydration, and is experimenting with coconut oil for nutrition.
"My golfer dad was shooting his age at 80 and made it to 96," he says.
Suter organised a chute of honour for Smith at one finish line.
"There aren't many events where you won't see Brian ... quietly shuffling across the finish line around prizegiving time. He does extraordinary things ... he fearlessly takes on impossible terrain and incredible distances and never requests any special compensation.
"I'll always shout the virtues of this amazing man."