A 1131sq m home will now be built on the site.
Approved plans for the home claim it will be "a modest country house/gentrified farmhouse which gives the appearance of having grown over time".
It will have a walled garden, orangery, basement cinema and five bathrooms, as well as space for five cars and a quad bike store.
A 44-year-old neighbour said the demolition was "typical Clarkson", adding: "Before there was a big old farmhouse but there is little to be seen of that now. Apparently they sent some of the villagers letters telling them something was going to happen. Any normal person would get in people to take it apart conventionally, but he blows it up. He blew up the entire house.
"All that's left is a huge pile of Costwold stone."
Another neighbour who saw the explosion said: 'All you could see was grey smoke each time they blew part of it up, but it was very loud in my house.
"There was a funeral being held at the village at 2.30pm, which was still on when the explosions started."
In 2013 Clarkson was granted planning permission for a six-bedroom house with a swimming pool, orchard, croquet lawn and tennis court.
But he submitted new plans in May to the same council. The new country pad will be made of rubblestone laid in lime mortar with ashlar stone dressings.
It will be 12 times the size of the average UK new-build, but won't have the pool or tennis court of the original plan.
But it will have a basement cinema, an attic games room, a walled garden, orangery, room for five cars, a yard for horses, six bedrooms and five bathrooms.
Clarkson infamously punched Top Gear producer Oisin Tymon at a hotel in North Yorkshire after allegedly being told he could not order steak after a day of filming.
The BBC sacked Clarkson from Top Gear in March last year, prompting co-presenters James May and Richard Hammond to quit the show.
Clarkson, May and Hammond signed up to work with Amazon and Top Gear was handed to Radio 2 presenter Chris Evans, who has also now left the show.