Mr Barclay, 25, said he was focused on doing his job and representing Clutha-Southland.
"We've got over 60,000 people in this electorate and I've got to do my best to be seen as a credible, accessible voice by every single one of those people."
The MP would not comment on reports which alleged that he had made secret recordings of a staff member, saying he could not discuss employment issues.
Prime Minister John Key backed the MP yesterday, saying that it was not unusual for staff to change when a new MP came in.
He said that he had not spoken to Mr Barclay about the departures at his electorate office and he did not have any concerns about the MPs behaviour.