But after another student laid a complaint about him, he was shifted to an unnamed Catholic school in the South Island, apparently without being punished.
"I fail to see where a geological difference can be the reason a pervert is cured," Cleary said in his statement.
Cleary said he did not address his abuse for 68 years because of shame. There was also no one to confide in, he said.
It was not until homosexual law reform in 1986 that he laid a complaint with police, who took no action.
He also made two complaints to the Society of Mary. One of the complaints was finally upheld in 2009, after Durning had died.
The hearings on faith-based institutions are focused on redress for historic abuse. Tina Cleary said her father never sought compensation.
"All Dad wanted was for Minto and Durning to be taken down," she said, referring to the pictures on the walls of the school.
After a complaint against Durning was upheld in 2009, they asked the school to remove his picture from St Patrick's.
Tiny Cleary said they received a legal letter from the school, which "made Dad feel worse". The picture was eventually removed but it "took years", she said. It was not known if Minto's picture was still on the wall.
His daughter said the abuse had a lifelong impact on her father. He had low self-esteem, suffered from depression, struggled to trust people and was always "waiting for someone to hurt him".
But he did not want to be defined by the damage done to him. He was a farmer, a writer, and a rugby lover. He had a "high regard for what was fair and was just", and protested against apartheid in 1981.
"Dad was really courageous and even in his death he was courageous. He was a giant in his death, he was 10 times the size of himself.
"He would want his words to be spoken so that it doesn't happen to other children.
"That's why he wanted the photos of those men in the school to come down, so that other children don't honour dishonourable men."
