"Everyone is devastated. We can't believe it - it's like losing a brother," Moya Hewson said.
It should have been a relatively easy one-day tramp to the hut for lunch before the 11-member party returned to the road.
The tramp to Nikau Hut was originally planned as one of several over the weekend.
Mr Brown was "tootling along" towards the rear of the strung-out tramping party when the accident happened. Some had already reached the hut.
The group activated an emergency locator beacon and most of them waited by the river for help to arrive.
Two of the party started walking out to raise the alarm.
Distressed club members were flown back to their cars by the Gisborne-based Eastland Rescue Helicopter while a second Whakatane-based helicopter lifted out Mr Brown's body late on Saturday afternoon.
Eastland Rescue Helicopter pilot Neil Dodds said he settled the helicopter at the bottom of a "tight little canyon", about 200m from the hut and 50m below the rock Mr Brown hit. He then took the tramping club members back to their cars.
He said the elderly trampers were a bit stressed. "They were not in a happy frame of mind and did not want to be where they were."
Mr Dodds said Saturday's mission highlighted the importance of carrying an emergency beacon.
Mr Brown described his love of tramping to the Bay of Plenty Times last year. He said manoeuvring through dense bush and over rocks with a heavy pack for days may sound exhausting but there was nothing like it.
"It's hard, it's hilly and then you get to the hut and soak up the fire and put your meal on ... you think you have died and gone to heaven."
Mr Brown got the tramping bug when he retired to Hahei in the Coromandel and later became a stalwart of Tauranga's Pack 'n' Boots Club after he and wife, Iris, shifted to Tauranga 13 years ago.
Mr Brown, described as fit, had been a very active club member and organised a series of major tramps to the South Island and Stewart Island.
His death has been referred to the coroner.