An Otago Daily Times investigation had since shown the Catholic Church moved Murray to Australia when his offending was first revealed to then-bishop of Dunedin John Kavanagh in 1972.
He was later allowed to return to New Zealand and resume public ministry in the North Island, where more victims have since emerged, before being asked to retire as fresh allegations emerged in 1990.
He had retained his title as a priest, even after his eventual conviction, until yesterday.
Two of his Dunedin victims were relieved he had finally been defrocked.
One of the men, who was an 11-and-12-year-old altar boy when abused by Murray from 1968-70, said his first reaction was "a sigh of relief''.
''This has been such a terribly long time coming, and for survivors of Magnus Murray this has been playing on our minds.
"It's just taken so long."
The second victim, who was 10 when Murray began abusing him, was also pleased to see a dark chapter in his life closed.
The impact of the offending had derailed his mental health, job prospects and relationships, and "ruined [his] life".
"Especially for my kids - when they were young I was just a zombie in those days."
Bishop Lowe, in a statement, said the defrocking had been "a complicated process because of Murray's age and health".
In recent years he has been living in retirement at Mercy Parklands, owned by the Catholic order Sisters of Mercy, while receiving care for dementia.
"Sadly, his removal from the priesthood does not take away any of the harm that Max Murray's criminal abuse inflicted on those he offended against.
"I hope this action may bring some sense of justice being done to those who have been abused," Bishop Lowe said.
The Catholic Bishop of Dunedin, the Most Rev Michael Dooley, echoed those sentiments, saying he hoped the move showed the church was committed to justice for survivors.
"Sadly this does not take away any of the harm that was caused by the criminal abuse to the victims [and] survivors by Max Murray.
"I hope it may bring some sense of justice being done to those who have been abused."
Network of Survivors of Abuse in Faith-based Institutions and their Supporters spokesman Murray Heasley was pleased by the defrocking, but questioned why it had taken so long.
"It is a deep and abiding tragedy that Murray was not defrocked in 1972 ... and that it has taken half a century to finally do the right thing by this man's many victims.
"This should not be necessary. The officials of the Catholic Church of New Zealand have to move from reaction to action on survivor demands for redress. This cancer has to be addressed."
