"Some of these incidents have an ugly element that most New Zealanders would find repugnant," Mr Bush said.
Last week a tourist driver was punched in the face and had their keys taken off them, after being pulled over by another motorist in Greymouth.
On Sunday a Christchurch driver filmed a tourist crossing the centre line in Lindis Pass before snatching his keys at Omarama.
Mr Bush said while there may be grounds to remove a driver's keys if there was critical or immediate danger, it was not a licence for people to take the law into their own hands.
"Police are the ones best placed to deal with poor and dangerous driving and anyone who witnesses driving behaviour that puts lives at risk should call 111 straight away."
Mr Bush said police were now patrolling areas popular with tourists in an effort to keep roads safe across the country.
Today, Associate Transport Minister Craig Foss announced the Visiting Drivers Signature Project (VDSP) would be extended to the West Coast.
"The government recognises that many people are concerned with poor driving behaviour on challenging roads in and around popular tourist destinations, particularly in the lower South Island.
"That is why we are extending the VDSP to include the West Coast -- an area that attracts a large number of tourists," Mr Foss said.
He said a range of safety improvements on state highways in Otago and Southland would also be fast-tracked for completion by July 1.
These improvements include an additional 50km of centre-line 'rumble strips', 140km of no-passing markings and 200km of highway marked with 'keep left' arrows.
"This work will improve safety for all road users, including the increasing number of overseas visitors choosing to explore our country by car.
"Every death or serious injury on our roads is a tragedy and these tragedies can be prevented by improving safety in every part of the transport system -- vehicles, speeds, road users and the roads themselves."
The safety improvements announced today would be in addition to a range of measures already in place in Otago and Southland, including 564km of edge-line rumble strips, 1800km of highway marked with 'keep left' arrows, 4755 curve warning signs and 165km of safety barriers.