Much has changed since 1986, when the Princeton philosopher Harry Frankfurt published an essay in an obscure journal, Raritan, titled On Bulls***. Yet the essay, later republished as a slim bestseller, remains unnervingly relevant.
Frankfurt’s brilliant insight was that bulls*** lies outside the realm of truth and lies. A liar cares about the truth and wishes to obscure it. A bulls****** is indifferent to whether his statements are true: “He just picks them out, or makes them up, to suit his purpose.” Typically for a 20th-century writer, Frankfurt described the bulls****** as “he” rather than “she” or “they”. But now it’s 2023, we may have to refer to the bulls****** as “it” — because a new generation of chatbots are poised to generate bulls*** on an undreamt-of scale.
Consider what happened when David Smerdon, an economist at the University of Queensland, asked the leading chatbot ChatGPT: “What is the most cited economics paper of all time?” ChatGPT said that it was A Theory of Economic History by Douglass North and Robert Thomas, published in the Journal of Economic History in 1969 and cited more than 30,000 times since. It added that the article is “considered a classic in the field of economic history”. A good answer, in some ways. In other ways, not a good answer, because the paper does not exist.
Why did ChatGPT invent this article? Smerdon speculates as follows: the most cited economics papers often have “theory” and “economic” in them; if an article starts “a theory of economic...” then “...history” is a likely continuation. Douglass North, Nobel laureate, is a heavily cited economic historian, and he wrote a book with Robert Thomas. In other words, the citation is magnificently plausible. What ChatGPT deals in is not truth; it is plausibility.
And how could it be otherwise? ChatGPT doesn’t have a model of the world. Instead, it has a model of the kinds of things that people tend to write. This explains why it sounds so astonishingly believable. It also explains why the chatbot can find it challenging to deliver true answers to some fairly straightforward questions.