It follows research that these flavourings, which are taken into the lungs within vapour from the aerosol, contain cancer-causing chemicals including formaldehyde.
And it comes as scientists at the University of Salford - who found e-cigarette flavourings such as butterscotch and menthol risk lung damage by killing off bronchial cells - call for better safety checks.
The danger to sperm is believed to come from chemicals in the flavourings such as coumarin, which is a cheaper version of cinnamon bark and commonly found in flavourings sold in the UK and made in China.
Lead author Dr Helen O'Neill, who presented the findings to the British Fertility Conference in Edinburgh yesterday, said the results were "shocking".
"In terms of motility, progression and concentration of sperm, there was a detrimental effect," she said. "E-cigarettes are promoted as the health alternative to smoking, the healthy thing to do.
"Vaping is less harmful than conventional cigarettes, but nonetheless they are not without their harmful effects."
There are 7,000 different flavours of electronic cigarettes, but those tested were the two most popular - cinnamon and bubblegum - and 'plain' devices just containing propylene glycol, a tasteless liquid used in the production of the vapour. Sperm samples were taken from 30 men, tested with concentrations of the flavours similar to the average intake for casual and more habitual e-cigarette users.
Researchers found sperm exposed to the highest concentration of flavourings moved significantly more slowly, while their swimming speed was affected. The biggest impact came from the cinnamon flavour.
The men, who were undergoing IVF but had healthy sperm, were not able to use the devices directly, but the flavourings were inserted directly into the sperm in the concentrations they would be exposed to. No nicotine was included. A second experiment looked at how mice reacted to being exposed to the flavourings - it found cells in the testes were killed off by the chemicals in them.
Bubblegum had the worst effect, with the highest number of dying cells in the testes tissue.
Dr O'Neill said these chemicals may harm men's fertility through the toxins they produce when heated in e-cigarettes.
Many flavourings are only regulated as foodstuffs, based on being consumed rather than inhaled.
Dr O'Neill said: "There is very little regulation before they are allowed onto the market."