Mr Deakin could not give an expected completion time, but that they would just "keep going".
The fringe buildings were coming down before work on some of the bigger buildings would start in a few weeks time, he said.
"Hopefully locals will be happy to see work starting. It might be a work in progress."
Gary Deakin who owns Hastings-based company Deakin Transport Ltd could not be reached for comment.
Central Hawke's Bay Mayor Peter Butler said he thought locals would be "very happy" work had finally started and said he had even asked them in his newsletter to deliver scones with jam and cream to the contractors.
"We've always wanted the site cleared, from day one, but it's been in private ownership."
Mr Butler said it was "the best thing to happen in Waipukurau in the last four years."
The Mayor thought a green patch, or a retirement village could be good options for the site.
Council chief executive John Freeman said contractors were liaising with Work Safe New Zealand to dispose of the asbestos in the building's roof.
Work Safe New Zealand asbestos disposal guidelines require contractors to contain and label all asbestos before disposing of it at an approved refuse site.
Waipukurau resident, Hayley Wakefield said it was good to finally see some work being done at the site.
"It's a rather large area that could benefit a lot of people, whether it be housing, or a park, just anything."
Miss Wakefield said getting rid of the buildings and creating a park could be good but she said there was also a need in the area for rental properties.
Pearl Pene, who worked at the hospital as a matron for some years, was pleased it was being turned into something.
In 1879, Waipukurau Hospital was Hawke's Bay's second hospital to open after Napier Hospital which opened in 1860.
In 1990, the hospitals in Napier and Dannevirke closed. Waipukurau Hospital closed in 1999, leaving just one in Hastings, Hawke's Bay Hospital, to serve the entire region.