"The suggestion that any MP would go to an event like this is disturbing. If these allegations are true, David Cameron should sack him."
Oxford-educated Mr Burley, who moved to England with his parents at the age of six months, has apologised for the offence the group caused.
"There was clearly inappropriate behaviour by some of the other guests and I deeply regret that this happened. I am extremely sorry for any offence that will undoubtedly have been caused."
Mr Burley had been seated next to a guest who raised a toast to the party's organiser and "the ideology and thought process of the Third Reich".
The group later moved on to a pub where members of the group put on German accents and chanted "Mein Fuhrer!" and the names of senior SS officers Heinrich Himmler and Adolf Eichmann.
One of the members of the group joked that Mr Burley - elected MP for Cannock Chase last year - was "the candidate for Berlin East".
The man in the SS uniform, reported to be groom-to-be Mark Fournier, could face charges under French law, which makes it illegal to wear Nazi uniforms and regalia unless for the purpose of a film, play or historical exhibition.
The controversy could also damage Mr Burley's ministerial hopes.
Mr Burley is a parliamentary private secretary to Transport Minister Theresa Villiers, who has Jewish constituents in North London and has campaigned against anti-semitism.
Private secretary roles are seen as a stepping stone to ministerial appointments in the British parliament.