Last week, Mr Goldsmith was photographed removing his own billboards from the Epsom electorate after they were put up without authorisation.
Staying out of the limelight does not appear to have stopped him gaining support, however, with National's endorsement of Act failing to sway many voters.
"If anything, the whole debacle has lowered my opinion of National," said a 70-year-old Goldsmith supporter.
"I don't like Act and I don't like being told who to vote for."
Said Susan, 61: "It's made me totally disenfranchised as a voter. I'll vote for Goldsmith because we want to get Act out."
In a rare point of agreement between Labour and National voters, Labour supporters spoken to said they would send their vote Mr Goldsmith's way to keep Act out of Parliament. "We definitely don't want John Banks," said one, Brian Murray, 63. "And unfortunately, voting for Labour in Epsom is just like throwing a vote away."
Tove Partington, 25, said of her vote for Mr Goldsmith: "Anything to give [National] one less crony in Parliament."
While true-yellow Act supporters made up a large chunk of Banks voters, National voters giving support to Mr Banks often did so with reluctance. "I'll be voting for Banks but I'll also be voting to change MMP," said a 53-year-old health worker. "I'd rather not have Act in there at all."
The street survey showed 31 per cent of voters were still undecided.
Many of these respondents admitted to being National supporters, torn between voting for Mr Banks or Mr Goldsmith.
With just four days left until election day, the Epsom seat is far from in the bag.
The High Court is set today to consider the legal status of the "teapot tape", and there are fears the furore over its contents may influence the many undecided Epsom voters.
But none of the voters surveyed indicated the tape saga would influence their votes.
"The media have taken a little grain of sand and created a Mt Everest," said Paul, a Remuera financier. "They seem to have their own agenda and are not at all in touch with the public."
The tape was repeatedly referred to as "trivial" and the media reaction compared to the behaviour of toddlers or children.
"Only the picky ones will be influenced [by the tape], and like donkeys they will be led," said Lorraine Crowther, 70.
Some said the Prime Minister's reaction to the tape showed he had something to hide and felt he needed to be honest with the public. "I definitely trust him less. As the leader of New Zealand he needs to be held responsible for anything he says," said George Xhao, 50.
Jo Hill, 39, said her opinion of Mr Key had dropped because of the tape debacle, but she would continue to vote National as she did not see a better alternative.
Most, however, simply saw the tape as a distraction.