The boxing was there at the beginning, out in Mitimiti, on the coast past the pub at Panguru which Charlie now owns.
Even now, 66 years after Charlie entered this world, it's a rattling ride along metal roads before winding to a halt amid sand dunes and houses. It's an isolated stretch of west coast, about as distant from everywhere else as anywhere in the North Island.
The farm on which he had grown was a Maori Affairs farm - one of those which soaked up the Maori workforce liberated from fighting World War II.
"We were just slaves to the system in those days. They called them rehabilitation farms. They weren't rehabilitation farms. They were slave farms. It was a slave labour f**king camp."
Boxing came with childhood. His father, Manuel Dunn, was a Maori Battalion boxing champion, taken prisoner during World War II. While a POW, he sparred with German Max Schmeling, world heavyweight champion between 1930 and 1932, who served as a paratrooper in the Luftwaffe. Sparring partners got extra food - "the extras were potato peelings".
So he grew up with it. As a boy, he sparred with his older sisters in the milking shed.
Then Charlie left for school in 1961, bound for Hato Petera College. He'd left the Hokianga just once before, along with most of Panguru, when the Queen came to Kaitaia. Pretty much everybody in town piled onto the back of a truck for the journey to see the monarch.
"They thought of the Queen as their own. Everybody had a photo of the Queen and King in their home. I think it was driven into their heads that she's going to do everything for us - not knowing she was taking the land from under their feet."
There were 10 other children in the family - there were four younger brothers and six sisters, some older. "I was the only one that was sent to college. My parents couldn't afford it."
The journey to school began on his father's tractor, along the northern Hokianga harbour to the ferry, which took him to Rawene. A boy of 14, he caught the bus to Kaikohe and then a train to Henderson. He stayed the night with friends and was taken to school the next day.
The discovery of the wider world was bewildering. "I'd found my way through the bush better than this," he said. "For a country boy, it was a bit of 'where the hell am I'."
The gloves came on after dinner and Charlie was, just a bit, back on familiar territory. Father Ryan, the Irish priest, took boxing training. "We'd make a mess of ourselves - all the stroppy ones would put the gloves on."
Hato Petera College was a revelation. "I loved school."
And then work. There was the glass works, then another factory then DB Breweries. Aged 18, he assured his employers he was older to get past the minimum age requirement. That came unstuck when he boxed and beat the Auckland heavyweight champion. His boss called him into the office the next morning, congratulated him on the result which was printed in the morning's Herald, pointed out that the article mentioned he was aged 18 and then fired him. "That was 96 a week. I'd been earning 13 a week before that."
There was league as well - that amazing game at Carlaw Park when he ran home a record-breaking seven tries. Look at these, he says, lifting an album of clippings onto the bar table. He was a stunning young man, captured ball in hand and feet in the air - the Hokianga flyer.
He had to give the league away in 1972 when his boxing turned professional. With the gloves on, he represented New Zealand twice, and when he wasn't in the ring he was blazing a trail across league fields.
As a professional, it all became about the boxing. And in those days, pretty much everything he wanted to do had little to do with going home to Hokianga. It was all about the fight - even trying to join the army to go fight in Vietnam. "I'm glad I didn't - there were five guys from here who went and only one lived." The chemicals sprayed on the jungle were brought home in their bodies, killing them young.
He was 34 and heavyweight champion when he went back up north. His boxing might have got him sacked when he was 18 - now the breweries and cigarette companies were falling over themselves to get a piece of Charlie.
He took up an offer to manage a pub in Whangarei and spent 15 years there, with wife Jan Edmunds, who he'd married in 1969. As the gloves came off, the league came back. All Black Joe Stanley remembers Charlie coaching him for a year, with the coach putting himself through the wringer as much as the players. "He'd lead from the front and he made sure all his charges worked as hard as he did."
There were some "unconventional" methods, he recalls. There was "a bit of a nudge" for some of those who were a bit rougher around the edges. But then that came back to those Charlie would draw on to the field, extending his reach to pull in those who would be otherwise excluded. "He had a lot of people who used to go around to his place. He used to take a lot of them under his wing - if they were needing a bit of assistance or were going off the deep end. For Charlie, it wasn't anything he went out to do consciously. It was just the sort of guy he was."
And that became life, along with the nine children he and Jan had of their own.
Charlie recalls: "We started looking after live-in boys. Mums and dads who didn't want them. Some who came to Whangarei and got lost." He took over a pie cart at night: "We had all these young boys who would come at night, scrounging around for food. I'd feel sorry for them and started taking them home."
For some there were jobs, for others it was sport. Always it was direction - and usually not so much a place to go as much as a place to be. Once the Dunn household gave them a place to be, they could work out the rest themselves."That would be one of my best memories."
Then there was the return to Hokianga, where Charlie took over the Panguru pub and where he's sitting to tell this story. "I had to make a lot of adjustments for a place I couldn't wait to get out of when I was 18."
There's industry here, now, which there hadn't been for years. His sons all live locally, working in forestry. The industry has liberated the northern Hokianga. "It's taken a lot of years to get away from dependency on benefits," he says.
The memories of the "slave" farm weigh on his mind - Mr Dunn works for himself. "There are no outside influences on me or my family. If it doesn't work, it's down to me.
"I think that's the issue with my people. They are too dependent on the pakeha man for handouts."
In his time back, he's also been a district councillor and sat on the school board. He also takes the kids for boxing lessons. It comes and goes, depending on the enthusiasm, which can ebb as he spends months teaching fitness.
Other times, they'll gather and Charlie will raise those big hands that pounded out a life.
"I think it was the thing that kept me out of jail. I still don't take any shit from anyone."
Your story
Do you have a great yarn or captivating tale to share? Fill in the online form here, email your stories, photos or video to nzhgreatest@nzherald.co.nz or share your story on social media using the hashtag #nzhgreatest.