Three excavators were partially engulfed on the outer edge of the slip. The extent of the landslide was more than had been anticipated.
One of the men, still in his machine, managed to get out of his cab but was visibly shaken, said the contractor.
"It was lucky that the call had been made to pull the workers off the slip before this happened."
The area had been prone to slips, and was being monitored and cleared on a regular basis - especially since the January slip.
But no one had anticipated the scale of the landslide that came down on Saturday around 5pm and it was only due to safety procedures that the situation was not more serious, he said.
"Safety is our first priority. The slip that blocked the road on Friday was being cleared when two spotters noticed the area had become more unstable and pulled everyone back.
"About an hour or two later it started to come down - then it really went with a 'whoosh'.
"Had people been working on the slip at the time, it would have been more serious. We all understand why the safety procedures were in place."
The slip is considered too unsafe to work on at the moment.
Instead, helicopters with monsoon buckets would drop water on the rock face today to pull down material that had not come down fully in the landslide.
The situation would be reassessed tomorrow, he said.
When it was considered safe to work, the clean-up would be a two-pronged approach with machinery working on it from both the Gisborne and Opotiki sides.
One of the problems facing contractors was trying to find a dump site that could take that amount of material.
Some deliveries to Gisborne have been affected, with Tip Top bread not making it to stores for three days.
Walter Findlay Ltd sales and distribution manager Clint Mengel said times like this showed how lucky Gisborne was to still have a bakery.
Mr Mengel said products like Vogels and Mackenzie bread from out of the region were still coming through on a daily basis from the south but deliveries were about three hours later.
"Better to get something a little later than nothing at all," he said.
Fonterra trucks were backed up at Makaraka for much of Saturday as drivers waited to see if the gorge highway would clear.
Milk is now going out through Taupo to Reporoa.
New Zealand Transport Agency acting Bay of Plenty state highway manager Andrew Scott said they would have a better understanding early this week as to when the gorge could be reopened once investigations had been completed.
"We need to be satisfied that the gorge is safe for drivers to use, so while we are clearing slip material we will also be determining the best course of action to stabilise rock face areas that could have been weakened by the slips," he said.
"We apologise to drivers for the disruption this has caused and we continue to assure those affected by the closure that we are working hard to safely reopen the Waioeka Gorge as soon as possible."
Gisborne mayor Meng Foon is urging NZTA to do its best to get the gorge open as soon as possible - stressing its importance as an arterial link between Gisborne and its northern markets.
The length of time to get the road open was of major concern to get freight to the area for supermarkets, a lot of it from Auckland, and get fuel in and stock to killing plants to the north.
Shortly the grapes would be harvested - another major user of the Gisborne-Opotiki link.
"Overnight courier services and our milk going to the northern milk factory will all be severely compromised," he said.
Mr Foon said he was also worried about the safety implications for drivers in having to travel an extra four hours to get north.