It is a different story at John St, where Mr Le Roux bought a small stucco-clad house.
He suspected it was originally a villa, and his hunch proved correct.
Working with heritage architects, he is rebuilding the house as a villa with garaging underneath.
"We have got to determine what the word demolition means," says Mr Le Roux.
Although he was required to seek a resource consent to demolish both buildings because more than a third of the properties would be lost, he said "sometimes you have to go back and change some of it to go forward".
Mr Le Roux, a South African who worked on residential and warehouse conversions in London before moving to New Zealand six years ago, said he was passionate about old houses and had never fully demolished one.
He acknowledges not everyone likes the work he does on properties such as Clarence St, but they do not understand his work.
"Some people see it [Clarence St] from the street and say I'm destroying heritage but I'm taking it back to what it originally looked like with slight tweaks."