Conditions were rough at the time, with 18 knots wind, gusting to 23 knots, a chop height of 0.6m and swell of 0.8m.
An investigation into the circumstances of the incident is ongoing.
A long-time friend of one of the deceased men described him "a good friend and brother".
Invercargill nurse Vini Kumar said she had known Richard Hiraman Jeet for 12 years and lived near him in Makarewa, north of Invercargill.
"I don't have any family in New Zealand and [he] was as close as family to us," Kumar told the Herald.
She said Jeet, whom she called Richard, was a strong source of support when her partner Ben McLean, a senior firefighter, died last year in a stag attack.
"Richard was like a brother to us and he would always come and help us on the farm."
"He was always there ... a very great man."
Kumar wasn't aware of the circumstances of Jeet's death, but said she had been to visit his family, who she described as in shock.
She said Jeet had sold his block of land in Makarewa about five months ago and was renovating his townhouse in Invercargill.
The close-knit Indian community was coming together to support Jeet's whānau and pay their respects.
"It's not a big Indian community in Invercargill so everybody is getting together ... it's something which is always there with them that their brother drowned, but in saying that, life does not stop there," Kumar said.
Bluff Community Board chairman Raymond Fife said he understood the boat involved was being used recreationally.
"It was rough conditions out on the harbour today. That may have played a part.
"It's tragic, and we feel for the family [of those who died]."
A rāhui has been put in place for the Bluff inner harbour, a post on the NZ Southland Hunting and Fishing Facebook page said yesterday.
"As a sign of respect and aroha to those who tragically lost their lives today, also our moana, Te Rūnanga o Awarua have placed a rāhui on the Bluff inner harbour.
"A traditional rāhui requires you refrain from taking kaimoana and pay respect within the notified area."
The week-long rāhui was in place in the area a direct line from Kanawera-Stirling Point to the Papakaha-Tiwai Point, and all the way up the harbour, including Awarua Bay, the post said.
- additional reporting NZ Herald