The real Jacinda Ardern. Photo / Mark Mitchell
A viral video that manipulated footage of Prime Minister Jacinda Ardern to make it look like she was smoking crack has reanimated debate over "deepfake" technology.
The video, which was viewed and shared thousands of times, showed a woman smoking from what appeared to be a crack pipe.
The PM's face had been superimposed using artificial intelligence. But the video, created for YouTube, was convincing enough to the many who shared it. It quickly spread to Twitter, Facebook and other platforms. Its creator's aim was to have a dig at Ardern over her Government's move to make smoking illegal for life for those who are under-14 today.
Think tank member Tom Barraclough, who co-authored a report on deepfakes for the NZ Law Foundation, is wary of over-reaction to the technology that could mute free speech, but also tells the Herald our politicians have been warned about the issue but are doing nothing (more on which below).
Another academic warns that the technology is now readily available - and that while it's still far from perfect, it's still good enough to go viral.
"Deepfake technology is becoming more prevalent and easy to access, as more people build more tools and make them publicly available," Auckland University research fellow Dr Andrew Chen warns.
Despite the technology becoming more sophisticated, experts like Chen can spot a fake.
Telltale signs of doctoring include blurring or pixilation, particularly around the mouth and eyes, badly synched sound, glitches, changes in lighting, gaps in storyline and irregular blinking.
But Chen warns, "A lot of deepfakes can be very convincing to the general public who aren't looking too closely. And if an expert is needed to analyse the video to find the telltale markers of deepfakes, then the disinformation can spread very quickly before the footage can be debunked."
Do our lawmakers need to act?
Chen points to 2019 report for the NZ Law Foundation on the rising deepfake threat. Co-authors Barraclough and Curtis Barnes said that while the fake video and audio clips were becoming more prevalent, they could be dealt with through existing legislation, including the Crimes Act and the Harmful Digital Communications Act - if they are properly enforced, and their application is clarified.
Special legislation to target deepfakes would put human rights and freedom of expression at risk, the pair argued. They said a Government could use such a law to clamp down on those who were using video effects for the purposes of political satire. They used the example of comedian Tom Sainsbury, who uses rudimentary AI video tools to imitate Kiwi politicians on social media. Barraclough and Barnes have since also carried out an assessment of Facebook for the NZ Super Fund.
This week, after the "Ardern" clip caused a stir across the Tasman, Barnes (who interprets the fake clip as cannabis rather than crack) said he and Barraclough stood by the findings of their 2019 report.
"The 'Jacinda Ardern Smokes Cannabis' video has not changed our minds," he told the Herald today. (The reboot that caused a kerfuffle in Aussie media over the weekend included a caption saying "Crack pipe? ... Such a role model").
"We were aware of it in May when it was published shortly before the referendum and election. As we see it, the Ardern video is more akin to a political cartoon, or satire. It's obviously fake, and plausibly could provoke interesting discussion about drug policy."
Barnes says a News.com.au article on the "Ardern" clip published over the weekend was similar to scare stories written four years ago.
"Since then, there have been no deepfake-related events with major strategic or political impact that we know of. As such, we find the article unjustifiably alarmist," says Barnes, who co-founded a think tank called The Brainbox Institute with fellow law grad and AI Forum member Barraclough.
But he cautions, "New Zealand law is still vague on whether it is a crime to use a deepfake to depict somebody as nude or having sex. Parliament is aware of this - it was raised by Brainbox, by Internet NZ, Netsafe, and the Privacy Commissioner. MPs have chosen not to address it."
Deepfake technology has often been used for fun. Actor Miles Fisher, for example, has recently gained 3.2 million followers on TikTok for his "DeepTomCruise" account, which features very convincingly rendered clips that have the Hollywood actor making various over-the-top statements. TikTok has ruled that the account - which is still active - falls within its parody rules.
But there's also political warfare side to the technology.
News.com.au did note that The Australian Strategic Policy Institute's latest report on the technology, titled "Weaponised deep fakes", takes a deep dive into where the problem is heading.
"Deep fakes will pose the most risk when combined with other technologies and social trends: they'll enhance cyberattacks, accelerate the spread of propaganda and disinformation online and exacerbate declining trust in democratic institutions," the report reads.
The authors say the "Russian model" of disinformation — the sharing of large amounts of propaganda — will benefit most in coming years.
"Online propaganda is already a significant problem, especially for democracies, but deepfakes will lower the costs of engaging in information warfare at scale and broaden the range of actors able to engage in it," the report reads.
"Today, propaganda is largely generated by humans, such as China's '50-centres' and Russian 'troll farm' operators. However, improvements in deep fake technology, especially text-generation tools, could help take humans 'out of the loop'.
"The key reason for this isn't that deepfakes are more authentic than human-generated content, but rather that they can produce 'good enough' content faster, and more economically, than current models for information warfare.
"Deepfake technology will be a particular value-add to the so-called Russian model of propaganda, which emphasises volume and rapidity of disinformation over plausibility and consistency in order to overwhelm, disorient and divide a target."
Justice Minister Andrew Little and Digital Economy and Communications Minister David Clark have been asked for comment.
Industrial projects dominate new supply, the study found, comprising 65% of new stock.