Professor Grandjean said: "You should definitely include fruits and vegetables in your diet, and that is especially important for pregnant women.
"The trouble is that, while the nutrients are crucial for the foetus, the pesticides can damage the development of the foetal brain.
"You get some pesticide exposure from domestic produce, but usage in the UK is somewhat restricted, so the main exposure is from imported conventional fruits and vegetables."
A study five years ago, led by Columbia University in New York, found children exposed to the pesticide chlorpyrifos in the womb had a lower IQ and memory and a thinner frontal cortex, showing it could cause "structural changes in the developing human brain".
This pesticide is banned in northern EU member states, but Professor Grandjean said British women could be exposed through imported fruit such as grapes.
There is also US research suggesting children are more likely to develop autism if they are exposed to pesticides by living close to farms spraying their crops.
The NHS already advises women to wash fruit, vegetables and salads to remove traces of soil, which may contain toxoplasma, a parasite which can cause toxoplasmosis and can harm an unborn baby.
They are also advised to avoid large fish such as tuna, which may contain mercury.
The latest report also suggests, in its advice from Professor Grandjean, that women should also consider avoiding tap water because it includes low levels of contaminants which can harm brain development, including arsenic, fluoride, and perchlorate.
He said: "I would advise to keep exposures as low as possible. After all, we have only one chance to develop a brain.
"And tomorrow's children ought to have the best possible start - the brain is our most crucial and vulnerable organ."
The report, titled No Brainer: The Impact of Chemicals on Children's Brain Development, says poisoning cases show at least 200 chemicals can enter the human brain and cause damage to the nerve cells, with children likely to be more vulnerable.
It adds that household dust also includes high levels of potentially harmful chemicals, such as phthalates used in PVC and similar plastics, brominated flame retardants and bisphenol A, which is currently being phased out in till receipts but is still used in food can linings.
People are advised to clean their homes regularly to stop the chemicals in dust from building up.
The report concludes: "We are always exposed to multiple chemicals, but regulations almost always assume we are only exposed to one at a time, even though numerous scientists have shown that chemical effects can add together in our bodies."
Elizabeth Salter Green, director of CHEM Trust, said: "We need action from governments to ensure that our children aren't exposed to any chemicals that could disrupt their brain development.
"In the meantime, it is a good move for vulnerable groups - such as pregnant women - to minimise their exposure to problematic chemicals. Eating organic where possible is one element of this, as is avoiding solvents, paints and other chemicals of concern."