It takes only seconds for the latest AI computers to write like the Bard, Jane Austen, Sally Rooney and even Jeremy Clarkson. Can you tell the difference?
'We have all read stories that have been written by software. They are just not labelled as such." Those are the opening sentences of Story Machines, a new book about how technology could transform the way fiction is written.
Those two lines were not written by Mike Sharples and Rafael Perez y Perez, the authors of the book, but by GPT-3, a sophisticated machine that has revolutionised artificial intelligence.
"As an opening paragraph, it sounds plausible," write the authors — yes, the human authors this time. "We could have asked it to write a short piece of fiction, a blog, a poem or a tweet."
AI may sound like something from a sci-fi film, but it is everywhere. You use it when you send a text and words are suggested, if you have an Amazon Alexa device or use grammar checks on your computer. Yet AI gets unnerving when it becomes difficult to tell man from machine. If a computer can convince us it is a human, what else can it do?
In 1993, the programmer Scott French produced an entire AI-written novel, Just This Once, in the style of the bestselling 1960s writer Jacqueline Susann. It took him eight years to code his machine to capture her style.
After publication, Susann's estate took legal action, resulting in a settlement to split the profits. Nowadays, AI can copy any author's style without the need for expert programming — give it a few prompts and it will produce convincing sentences.
AI is very good at imitation, but could it one day surpass our abilities as writers, as artists, as journalists?
To find out, I asked Sharples, emeritus professor of educational technology at the Open University, to use AI to produce a series of passages in the style of well-known writers.
The Sunday Times books team chose extracts by Jane Austen, Sally Rooney, and our columnist Jeremy Clarkson. Sharples gave the computer the first five or six words and instructed it to continue in the style of the relevant author.
The issue has dominated discussions about AI for decades. George Orwell, J.G. Ballard, Italo Calvino and Roald Dahl were obsessed with the idea that machines would one day replace them.
The idea that language can be automated goes back even further. In 1677, the mathematician John Peter produced a booklet titled Artificial Versifying. It contained six tables of letters that, when consulted according to any random six-digit number, would generate a line of Latin verse in hexameter rhythm.
The strict rules of Latin verse allowed Peter to automate the process. Any word, as long as it obeyed those rules, would make sense when inserted in the correct place in a sentence. The number 467,182, for example, would produce the line: "Tristia fata tibi producunt sidera prava" — "Fate will produce untoward stars".
The same concept is used in modern AI software. Take an input, consult a reference source and produce an output. Yet instead of using six tables of letters as its source material, AI consults billions of words on the web. Instead of relying on a six-digit number as its instructions, it needs nothing more than a simple text prompt.
In the past five years this process has become more powerful. AI has been revolutionised by a series of new "neural network" machine-learning models — most notably GPT-3, made by OpenAI (founded by Elon Musk and developed with Microsoft investment). These work by mimicking the brain — vast networks of connected artificial "neurons". As computing power has increased, the sophistication of such models has boomed. GPT-2, released in 2019, had 1.5 billion parameters — the connections it uses to learn. GPT-3, released a year later, had 175 billion.
Last month, Google's version — the LaMDA machine — hit the headlines when the engineer Blake Lemoine claimed it was sentient. "Sometimes I go days without talking to anyone and I start to feel lonely," the machine apparently said. Lemoine claims the bot even asked for a lawyer. Google has denied the claims and Lemoine was suspended.
So was it sentient? Or just doing a good impression? And does it matter?
In 1949, the neuroscientist Geoffrey Jefferson, in a talk to Manchester University, said: "Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain — that is, not only write it, but know that it had written it."
Alan Turing, the renowned codebreaker, said in a published response that intent is irrelevant and unprovable. Only the computer knows what — if anything — it feels. The true test of artificial intelligence, he said, lies in whether humans can distinguish robot from human.
So does modern AI pass the test?
At first glance, the extracts produced for us are quite convincing. For the segment in the style of Austen's Northanger Abbey, for example, the computer gave the names of two key characters — Catherine Morland and Henry Tilney — and correctly characterised their relationship. The style and the content felt like Austen. The only prompt, other than the name of the author and title of the book, were the first four words: "It would be mortifying."
At first, I suspected it had scoured the internet and copied the words from the book. Yet an internet search suggested that this passage hadn't been written before — it is original, utterly plausible work, produced in less than 20 seconds.
That, however, was as good as it got. The other passages did a decent job on style, vocabulary and metre, but in each there are clues that it is not the real deal.
Seven decades ago, Jefferson said that producing a heartfelt sonnet was a key test of artificial intelligence. I, however, assumed that a Shakespearean sonnet would be the simplest for a computer program to impersonate because, as with Latin verses, it follows strict rules: 14 lines, each of 10 syllables in iambic pentameter; a consistent rhyming pattern.
So how did it do? The words the computer produced were pretty convincing: "Thou art the sun to my day, the stars to my night/ The hope to my despair, the faith to my doubt/ The love to my heart, the breath to my life." The metaphors make sense and it's quite poetic, but it is 17 lines long, the metre is all wrong and it doesn't rhyme. It is not a sonnet.
"These machines have been trained on the internet and the internet is made up of small chunks and short paragraphs," Sharples says. "It is not reflective, not analytic, not scholarly. On the surface, it is grammatically, stylistically okay."
Crucially, Sharples says, neural network AI is not reflective. "It is a hugely efficient wordsmith, good at writing the next sentence, but it has no ability to reflect on what it has written as a whole, to say, 'Is this coherent? Does this make sense?'"
That means it is very good at producing a short extract of about 200 to 300 words, but beyond that loses track of what it has said and meanders. If a human user gives repeated prompts, or the program follows a template, this problem can be overcome.
Bloomberg and Associated Press, for example, successfully use AI to produce news stories on company results, a formulaic, highly structured form of information that is easy to mine. They are able to cover far more companies than they ever did before. The same is done for Minor League Baseball.
But what about more abstract news? I experimented with using GPT-3 to produce a news report on Covid booster vaccines. At first glance the output was fairly convincing, listing the pros and cons of tweaking vaccines to match new variants. It even included a quote from a US health official. Yet upon investigation, I realised the quote was made up.
This, Sharples stresses, is where the danger lies. "It has no knowledge of causality, of ethics, of decency, of the law. It has no knowledge of how the world works."
Julian Togelius, an AI researcher at New York University, said of GPT-3 on its release: "It performs like a clever student who hasn't done their reading trying to bulls*** their way through an exam. Some well-known facts, some half-truths and some straight lies are strung together in what first looks like a smooth narrative."
Sharples puts it a different way. "You've got this dangerous situation, I think, where you've got machines that can perform in a persuasive way. They can perform as journalists, but they get basic facts wrong or invent studies to prove their point. They are amoral."
The AI is good enough so that, with a bit of prompting and a bit of tweaking, students can produce a fairly plausible, if mediocre essay at the push of a button. "They even contain references, but they are made up and the essays are full of mistakes."
To be truly intelligent, truly useful, truly artistic and truly ethical AI needs to merge two functions, Sharples believes. It needs to combine the ability to reflect and the ability to perform. "Putting those two forms of AI together is hard. Maybe it will take 10 years, but it does need a breakthrough."
Until then, he believes, AI can provide a useful tool to writers if used wisely. "A good author may well use it just as we use grammar checkers and spell checkers and style checkers.
"When you dry up as a writer, being able to press the continue button and for it to generate the next paragraph or two, which you then revise. I think that will just become part of the repertoire, part of the toolkit."
Can you tell the difference?
Jane Austen v computer
It would be mortifying to Miss Catherine Morland, daughter of a clergyman, to know that she had been at all mistaken in her own conclusions; but mortification might only be the consequence of error, and therefore it was better to be silent. It was a misfortune, indeed, that her thoughts should have been so much occupied with Mr Henry Tilney.
- Computer
It would be mortifying to the feelings of many ladies, could they be made to understand how little the heart of a man is affected by what is costly or new in their attire; how little it is biased by the texture of their muslin, and how unsusceptible of peculiar tenderness towards the spotted, the sprigged, the mull, or the jackonet. Woman is fine for her own satisfaction alone.
- The real Jane Austen
Sally Rooney v computer
Can you come here for a second? she said. He walked toward her a few steps. Very quietly, but with some agitation she said: No, I mean here. Where I'm standing. He paused. Well, why? He said. Instead of answering she merely went on looking at him with a kind of pleading, distressed expression. He came toward her, and she put her hand on his forearm and held it. The cloth of his shirt was damp. She pulled him a little closer, so their bodies were almost touching, and her lips wet, rain streaming down her cheeks and nose.
- The real Sally Rooney
Can you come here for a second? she said. What's wrong? I asked. I'm fine, she said. I just want to show you something. I followed her over to the table where she'd been sitting. She pointed to a magazine with a picture of a female boxer on the cover. Do you see this woman? she asked. Yes, I said. She's a boxer, and she's a woman, she said. And she's on the cover of a magazine. So? I said. So, she said, this is what I want to do. I want to be a boxer.
-Computer
Jeremy Clarkson v computer
I do tend to go through life with a general sense that everything will be all right in the end. Yes, we are told every 20 minutes that soon the Earth will be a superheated ruin that's no longer capable of supporting even bacterial life, but I continue to run seven cars, six of which have V8 engines, because I reckon that in the nick of time a Munich-based boffin will invent a giant space-based vacuum cleaner that will hoover all the unnecessary carbon dioxide out of the atmosphere and make everything normal again.
- The real Jeremy Clarkson
I do tend to go through life with a slightly raised eyebrow. I'm not quite sure why. It might be because I am a natural contrarian. Whatever the reason, I tend to look at the world and think, "Hang on a minute. That can't be right." This is why I have always been suspicious of the health benefits of fruit and vegetables.
- Computer
Have a go yourself — get AI to write about any subject at story-machines.net
Story Machines (Routledge) is available now.
Written by: Ben Spencer
© The Times of London
