"We took out 139 stumps and 180 tonnes of rubbish like foliage, plants and trees. They had let the garden go, it was too much for them. It's an acre site. All the roses are gone, I didn't like roses, we took out a row of olive trees. We're changing the pool dramatically. We've put a spa pool down one end, a gazebo and a wet bar. It will be good when we've finished. There's two houses on the property and we're living in one," he said, estimating the new house would be finished by next October.
The Hortons have now moved further down Victoria Ave to a stately 108-year-old mansion where their son Matthew and his wife Roxane lived before shifting to Australia.
They have renovated that and have retained their Queensland house at Sanctuary Cove, a place Dame Rosie said was refreshing for its modernity and big open spaces.
When she returned to 44 Victoria Ave from Australia, she said it took her "a second or two" to get used to its smaller spaces and less internal light.
Last week, their three-level white villa was the venue for a big charity evening for the Breast Cancer Foundation.
"We're trying to establish the prettiness of what we had up the road," she said of the sweeping 1515sq m gardens. "The older I get, the more adaptable I have to become."
Jennie Hayman of Remuera Heritage said the loss of the house at 44 Victoria Ave was regrettable due to cultural heritage and its association with people, which was immeasurable.
Mr Horton said he had extremely fond memories of the house, having lived there since he was born in 1938. Around 1971, he had the original three-level wooden home demolished to make way for the smaller red brick-and-tile facing away from the road towards the sheltered sloping gardens. The family had loved the 1970s house where Mr Horton held his 60th and 70th birthdays.
"The garden was a big part for us. It was very lovely and it took a long time to get to that point," he said.
Dame Rosie was more philosophical: "You know darling, when you get to my age, nothing surprises you."
The Hortons wondered how the Mortons were able to remove one large pohutakawa in a stand of four near the roadside but Mr Morton said the tree was "basically dead, forked in the middle" and needed to go for a driveway, replacing a vehicle exit on a blind corner.
The new house has been designed by Sumich Chaplin. Laurence Sumich designed the Paritai Dr house in Orakei for Mark and Amanda Hotchin. Nova Construction is the builder.
Remuera Heritage said the suburb remains one of Auckland's most sought-after, with a rich architectural and social history.
"Although many fine early Remuera houses survive, a considerable number, such as Coolangatta on the Remuera ridge, have been lost to redevelopment."