Late greats: why some brilliant ideas get overlooked

‘The stories of Jansky, Mendel and Bayes hold out hope to anyone who feels that the world has not quite caught up with them’. Photo / Getty Images

In 1928, Karl Jansky, a young radio engineer at Bell Telephone Laboratories, began researching static interference that might obscure voice transmissions. Five years later, after building a large rotating antenna and investigating every possibility he could think of, he published his remarkable conclusion: some of the static was coming from the Milky Way.

Jansky's theory was eye-catching enough to be published in The New York Times but scientists were unimpressed. Radio signals from outer space? Surely they were too weak to detect. Jansky's ideas were largely ignored for about a decade. He died at the age of 44. Thankfully, he lived long enough to see his ideas blossom into the field of radio astronomy.

Jansky's story resonates with us: we all like the idea of the researcher who is so far ahead of their time that it takes years for the rest of the world to catch up. Gregor Mendel's research into plant genetics is a famous example — published in 1866, it was only verified and taken seriously in 1900.

So what is it about an idea that delays recognition? One view is that brilliant ideas are overlooked when delivered by obscure messengers

An even more striking case is that of Bayes' rule, developed by Thomas Bayes in the 1740s, apparently abandoned even by Bayes himself, then rediscovered and refined by Pierre-Simon Laplace in the 1770s. Bayes' rule explains how to update previous beliefs in the light of new information. There's nothing objectionable about the mathematics behind it but what caused controversy is the idea that you can start with an arbitrary guess, then update it over time as new information comes in.

As Sharon Bertsch McGrayne relates in The Theory That Would Not Die, this was regarded as nonsense by most statisticians for much of the 20th century: garbage in, garbage out. It is now respectable again.

To my mind, this revival is all to the good. Old-school statistical sticklers would have shrugged at a question such as "What is the probability that a respiratory virus causes a worldwide pandemic?" A category error, they would argue: probability describes long sequences of repeatable events, so there is nothing to be said about unique ones. Bayes' rule offers a pragmatic alternative: take an educated guess, then update that guess in the light of Sars, Mers and now Sars-Cov-2.

The stories of Jansky, Mendel and Bayes hold out some hope to anyone who feels that the world has not quite caught up with their brilliance. There is even a name for such cases, coined by Anthony van Raan of Leiden University: "Sleeping Beauties", scientific papers that receive almost no citations for years, before finding wide attention. (Some scholars argue that the term is sexist and prefer "delayed recognition".)

There are several examples in economics — for instance, Thomas Schelling's 1956 Essay on Bargaining was an elegant discussion of the role of credible threats and binding promises in all manner of negotiations. But it was not widely cited until Schelling won the Nobel memorial prize in 2005, well into his eighties. That gap was longer than Karl Jansky's entire life.

So what is it about an idea that delays recognition? One view is that brilliant ideas are overlooked when delivered by obscure messengers. Bayes, Jansky and Mendel were all somewhat detached from the scientific mainstream, but Pierre-Simon Laplace certainly was not. In 1970, the sociologist Stephen Cole published an analysis arguing that the obstacle tended to lie in the content of the idea itself, rather than the prestige of the scientist behind it. Ideas fell asleep for a hundred years because they were radical, or confusing, or both.

It is difficult to be sure. Two scholars of the field, Eugene Garfield and Wolfgang Glänzel, have argued that such examples of delayed recognition are so rare as to be hard to analyse. Studying papers published in 1980 from the vantage point of 2004, they looked for articles that were barely cited for five years, then subsequently took off. They found just 60 examples in 450,000 cases. There are plenty of examples of research that is barely cited; what is rare is their subsequent popularity.

Why, then, is this myth such a compelling one? One explanation, of course, is that we all love a story of the underdog who triumphs against the odds. Immediate and sustained success is as boring as immediate and sustained failure.

Another is that scientists themselves are fond of the thought that their ideas are under-appreciated. In an essay on delayed recognition, Garfield notes mildly that one historian of science, Derek Price, believed one of his own papers was suffering delayed recognition. It is easy to chuckle, but it is also easy to empathise.

Delayed recognition is rare. Much more common is for people simply to reach their prime late in life. David Galenson is an economist who studies the creative output of musicians, artists, directors and others. Galenson has found that while it is quite possible to break through as a radical young conceptual artist, there are many examples of "old masters" whose later works are more admired than their youthful ones.

We all need to be able to hold on to the idea that the best is yet to come. But it is too tempting to hope that what we have already produced will, one day, be recognised for its brilliance. Good things do not come to those who wait, if waiting is all they do. It is wiser to get back to work and make something better.