Instead he was sacrificing every fibre of his being for the pursuit of Olympic glory under the tutelage of the understated yet hugely renowned Andrew James Duncan Laing.
Laing's association with the Olympics began forty years earlier, back in 1956, when he captained the New Zealand surf-lifesaving team at the Melbourne Games. Surf Lifesaving was a demonstration sport back then, but his appearance paved the way for what would become the most enduring family Olympic legacy in New Zealand history.
Twenty years after Melbourne, Duncan Laing, as he was known, was coach of the New Zealand swim team for the 1976 Montreal Games, a decade after he took up coaching at Moana Pool in Dunedin. His chargers included
Fast forward another twenty years and the shy, clean-living Loader was blitzing the competition in the Atlanta pool under the guidance of Laing, who had told Loader's mother a decade earlier that if he could harness this little nuisance they could have a champion on their hands. And so it proved, with the Laing/Loader relationship now firmly cemented in New Zealand Olympic folklore.
But the story doesn't end there. Duncan Laing passed away in 2008, (I remember covering the public service for Newstalk ZB and being quite entertained by the legion of stories, from the thousands of local kids he coached right up to the greatest swimmer of them all, Michael Phelps, who was schooled by Laing at Moana Pool in 2003), and now his son Graeme is picking up where the old man left off.
Twenty years after Atlanta (you do see the pattern here, right?) Graeme has been charged with getting Lauren Boyle and Matt Stanley up to speed to take on the world in Rio.
He says his days growing up in Laing household pretty much meant dividing your time between swimming and rugby. Graeme was a handy prop forward while Duncan was one of Laurie Mains' selectors and a title winning coach with a number of clubs, including the world's greatest, Otago University.
Graeme Laing says he's proud to be part of the family that's carried on with swimming right through.
It's been hard work for New Zealand since Danyon's double but he hopes the knowledge of past Olympic campaigns he and his brothers, Ricky and Stephan, gleaned from their father will now be taken heed of by the current generation of New Zealand swimmers.
And while we don't know where the 2036 Olympic Games will be held, I wouldn't mind betting there'll be a Laing involved somewhere.