The younger three, Rosalind, Alan and Tia, avoided orphanages but endured a tough upbringing nonetheless.
Their traumatic upbringing haunted the children and Caroline, Wendy and Rosalind took their own lives.
Despite the way they were treated as youngsters, Janine and Tia continued to look after their mother runtil she died. Tia described herself as a confidante for her mother after the three suicides.
However, despite the sisters' voluntary efforts, they were mostly left out of their mother's will which left 93.5 per cent of her $2.33 million farm to Alan.
This created a wedge between the siblings and the sisters launched a case in the Family Court.
The court found in the sisters' favour and decided the division of the farm should be in equal 33.3 per cent shares.
Their brother appealed that decision to the High Court on the grounds the judge based her findings on "fairness" which, in terms of the law, a judge is not allowed to do when determining will disputes.
In her decision, Justice Katz of the High Court stated The Family Protection Act 1955 allows for courts to do "no more than the minimum to redress a testator's breach of moral duty".
"Beyond that point the testator's wishes should prevail, even if the individual judge might have seen the matter differently. The court's power does not extend to rewriting a will because of a perception that [the will] is unfair."
Justice Katz found the Family Court judge relied too leniently on what was fair in treating all her children equally, thereby ordering the will be divided equally.
She wrote that Alan had weight in his argument, however Ormsby "clearly owed all three of her children a strong moral duty, particularly given her shortcomings as a parent and the serious deprivations the children suffered during their childhood (in respect of which Janine appears to have suffered the most).
"People are at liberty to do what they like with their assets and to treat their children differently, provided that they make such provisions as are necessary to discharge their moral duty to those entitled to bring claims under the Act."
The judge must take into account the children's financial position and their entitlement to be recognised as a member of the family, Justice Katz wrote.
She found that as per their mother's wishes, Alan should still have a majority, giving him a 45 per cent share, and Janine and Tia each receiving 30 and 25 per cent respectively.
Alan appealed the High Court decision, which was issued in November last year.
In its decision released on Monday, the Court of Appeal dismissed Alan's application, stating that the sisters were in financial need and the dividends were on par with case law.