Things soured when last year, Joseph was told the company had "made a loss on the books" over the previous two financial years, which he found hard to understand. "We then had a major argument and have scarcely spoken since," Joseph told the FWC.
In March this year, Paul called Joseph to a meeting with Chris, who works as a graphic designer and store clerk at the company, and administration manager Arnaz Ardi.
The meeting was to discuss "long-term issues arising and impacting upon the company as a result of the manner by which Joseph had been conducting business", according to Paul's statement to the FWC, in particular "decisions Joseph had made to pursue markets and sell certain products at a heavily reduced price".
Joseph told the FWC they "just wanted to ask me to resign". "I think Paul had the intentions [sic] of getting me out of the way so he could further gain his power and shares," he said.
After Paul and Mr Ardi told Joseph he needed to cease selling products at near or below cost, Chris backed the pair up. Then things got heated.
"Chris, you are a useless f***ing fat slob that draws penises and vaginas all day and Arnaz you're a dopey f***ing South East Asian not worth feeding," Joseph is alleged to have said.
Under cross-examination, Joseph gave a slightly different version, conceding that he told Chris, "Christopher, you're useless, you dawdle around everywhere and have a habit of drawing genitalia."
He denied calling Mr Ardi a "dopey f***ing South East Asian", saying he simply told him, "Stay out of it, Arnaz you idiot."
The FWC heard that Paul then stood up and told his brother his "constant bullying and inappropriate racial remarks will no longer be tolerated" and that he needed to resign as director.
In his statement, Chris said Joseph then told Paul that "if you ever ask me to resign I am going to smash you in the face".
Mr Ardi alleged Joseph "often made improper comments and inappropriate references to my race and culture which I chose to ignore, although I was not happy about it".
"He often says, 'You and your Indonesian cousins are f***ed', 'All you South East Asians are backward compared to the rest of the world'," he claimed in his statement.
Another employee who worked in the company's warehouse, Yong Xie, told the FWC Joseph was often extremely rude to him, and had called him a "f***ing Chinese c***".
The FWC found that in April, during an altercation in the warehouse office, Joseph put his finger in Paul's face and said words to the effect of, "I'm going to take you outside and I'm going to bash you."
That day, Paul sent his brother a termination letter. "Your behaviour towards staff is objectionable," it said.
"You have on many occasions been verbally abusive and used obscene language in circumstances where it is inconsistent with your duties including but not limited to making threats of physical violence to the managing director.
"This decision was not made lightly but was necessary given your recent conduct and attitude to the company and its affairs."
The FWC heard how Joseph's behaviour had caused friction in the company since before their father had left the country.
In a 2001 letter from the company's accountant, written on behalf of Fred Essey, Joseph was told that "violence, the threat of violence, abusive language and yelling are not acceptable parts of a business environment".
"All of your brothers receive your criticism and verbal abuse and of recent times this has reached a level, I am informed by your father, which is unacceptable and threatens the continued involvement of your brothers in the business," the letter said.
"Quite frankly they don't enjoy going to work and being abused by you.
"There have been outbursts of this behaviour from you on different occasions over the years and now it is becoming a behavioural pattern. Your father goes overseas tomorrow and fears that this will escalate during his absence."
In 2010, police were called to the premises after Joseph "punched Chris in the head and caused him to roll down stairs", according to Mr Ardi, who said Joseph was "held by the police" but the matter was ultimately dealt with in the family.
Joseph agreed he had been involved in a physical altercation with his brothers in 2010 but told the FWC they had attacked him.
"I have behaved towards staff the same way for 30 years as far as I am aware," he told the FWC. "I admit I swear occasionally, and so do others including Paul. I have neither increased, nor decreased, my level of swearing in recent years.
"As far as I'm aware, a certain amount of swearing around warehouses in Australia is commonplace and I do no more than this. I deny I made 'threats of violence' toward Paul."
On the 2001 letter, Joseph said his father "was like Paul, on a power trip as well". "They're the same," he told the FWC. "They just wanted to write a letter, get people to write letters and make up, just to get you to do what they want, otherwise they start getting accountants to write letters to threaten you."
Joseph said being sacked had left him to find a new career at the age of 53. "Becoming an employee at my age is next to impossible, so I am left to do such bits of consulting work as I can do," he told the FWC.
In his decision, FWC senior deputy president Jonathan Hamberger found the incident on 7 April "by itself constituted a valid reason for the applicant's dismissal".
"Indeed, it is hard to think of a more valid reason for terminating an employee's employment than threatening to take the managing director outside and bash him," he said.
"However, to this must also be added the applicant's abusive and racist remarks directed at Mr Ardi, Mr Xie and his own brother Chris.
"I note the applicant's comments about the difficulty he faces finding other work at his age. However, this has to be weighed against the appalling misconduct which led to the applicant's dismissal.
"I am satisfied that the applicant's dismissal was neither harsh, unjust nor unreasonable."
All-Fect and Mr Essey have been contacted for comment.