While most agreed AVs would be safer, one of the main concerns listed was that machines lack "common sense" needed to interact with human drivers.
A number of drivers surveyed also expressed reservations about the mix of human agents and AVs sharing the road, with human instinct and skill often trumping machine logic in dangerous situations.
"If there were only autonomous cars, however, I would maybe feel even safer. But this mix, I don't' like so much" one respondant said.
Mercedes has made the call that, if forced to chose to save the occupants or bystanders, the car will save the occupents, bypassing the moral and ethical conundrum of compuational valuing of human lives. A decision which isn't likely to inspire confidence in other road users.
"If you know you can save at least one person, at least save that one. Save the one in the car," said Mercedes technician Christoph von Hugo.
In tune with the survey's findings, the first self-driving cars to hit roads in Britain will be unmarked and indistinguishable from other cars to throw off drivers who might be tempted to take advantage and "test" the vehices.
Erik Coelingh, senior technical leader at Volvo Cars, told the Observer that one of the biggest fears in the industry is that other road users will drive erractically and slam on their brakes in order to "test" or "throw off" the autonomous vehicles.
Volvo is rolling out a pilot test by giving 100 self-driving cars to everyday road users in London - but they will look identical to another Volvo model.
"Just to be on the safe side they will all be unmarked cars. I'm pretty sure that people will challenge them if they are marked by doing really harsh braking in front of a self-driving car or putting themselves in the way," Coelingh told the Observer.
AVs are set to shake-up the insurance industry too - with Ceolingh stating that if the car is involved in an accident while in self-drive mode, even if the driver is reading a paper, Volvo is liable. Furthermore, with sensors and cameras covering the car, it will be far easier to determine whether the cause of the crash was a third party.
The trial with everyday drivers is set to begin in London in 2018.
A demonstration of the self-driving Volvo vehicles begins in Tauranga on November 17.