Ivy suffered her stoke on January 6, 2014 and she and her husband Fred were forced to leave their Forest Hill home and move into a Northcote rest home.
The couple had enough savings that they weren't forced to sell their house.
During her life, Ivy had made several wills, all of which stated her three children were to take equal shares from her estate, court documents show.
However, on January 13 just a week after her stroke, Ivy gave Stephen power of attorney for her property and welfare.
On the same day Stephen signed a listing agreement, seemingly without Ivy's knowledge, with Ironbridge Real Estate Limited to sell the Forest Hill property for $670,000.
Two days later Ivy signed a new will asking that the proceeds of a sale of the Forest Hill property would be divided equally among her children.
The rest of the estate was to go to Stephen and Linda, after Ivy had earlier made provisions for Raymond and daughter Andrea, who died who died on 15 March, 2015.
The family home was sold on January 24, 2014. The proceeds came to $646,507 and were deposited into a joint Westpac account under Ivy, Fred and Stephen's names.
After Fred died on October 7, 2014, and later Ivy the following year, there was just $168,464 left in the account.
The missing $586,704 had been withdrawn by Stephen, court documents show.
Ivy and Fred also held accounts at ANZ, and Stephen again used his power of attorney to draw funds from those accounts.
Up to Ivy's death, Stephen had withdrawn $338,480 from the ANZ accounts, resulting in the total amount withdrawn after being named power of attorney to more than $756,000.
Ivy had named Stephen and Linda the executors of her will, however, the pair continued to withdraw funds from the accounts.
A further $258,255 was removed from the ANZ and Westpac accounts after Ivy's death, court documents show.
Raymond also claimed the sale of the family home, done so "with unseemly haste", was sold below market value.
A registered valuer said the home was worth $750,000 at the date of sale, but was on-sold on March 6, 2014 for $795,000 and again on November 20, 2014 for $913,000.
When Raymond lodged proceedings at the High Court, and Stephen and Linda were requested to appear, it was noted that they had left for Queensland.
"There is persuasive evidence that Stephen and Linda were trying to avoid service," Justice Bell said in his ruling.
"I am satisfied on the evidence and in the absence of any explanation from Stephen that he breached his fiduciary duty to Ivy."