But ovarian removal immediately brings on the menopause, as the body's method for producing the oestrogen hormone is removed.
Dr Gillian Einstein of Toronto University, an expert in the way dementia hits women, said oestrogen has a protective impact on the connections in the brain - and when natural hormone production stops this protection disappears.
Her team tracked 133 women aged between 35 and 50, measuring their brain power with a series of tests for at least three years.
A third of the group had surgery to remove both ovaries, another third carried the BRCA-gene mutation but did not have the operation, and a third were healthy volunteers used as a comparison.
Dr Einstein's team found that the women who had lost their ovaries displayed declining scores in the tests. They did worse in word-recall and logical memory tests, some of them showing protracted decline for eight years after the operation, a drop not seen in the other groups.
But if the women had been given HRT pills - which artificially replace oestrogen - less decline was seen. Dr Einstein said the results suggested oestrogen, whether natural or artificial, is vital for brain health. And when it is removed, it may send women down a path of decline that ends in Alzheimer's disease.
Cancer experts want even more women to have their ovaries removed in their 30s and 40s, as they come up with new genetic tests to calculate someone's cancer risk many years before a tumour appears. For women at risk, removing the ovaries almost entirely eradicates the chance that they will develop cancer.
Interest in the procedure soared after Miss Jolie, 41, chose to have her breasts, ovaries and fallopian tubes removed after her mother died of ovarian cancer at 56.
Dr David Reynolds, chief scientific officer of Alzheimer's Research UK, said Dr Einstein's study was important but more research was needed.
Katherine Taylor, chief executive at Ovarian Cancer Action, said: "We welcome research that builds on our knowledge of BRCA and cancer prevention but urge women with BRCA gene mutations not to panic in light of this new research, which is in early stages."
Dementia's silver lining
Being diagnosed with dementia may have a "silver lining", the conference heard.
Scientists from Kentucky University who studied patients with dementia or mild cognitive impairment - often a precursor to the disease - found that more than half said their diagnosis gave them a new appreciation of life.
Others were glad to finally receive an explanation of why they had been suffering memory problems or confusion, in some cases for years.