Jacinda Ardern and Emmanuel Macron pledge to tackle algorithms driving people to extremism

New Zealand will spearhead an international effort to understand how the algorithms that sit behind popular social media websites can drive people to more extreme content, potentially radicalising them.

It is hoped the research will lead to companies improving their algorithms to minimise social harm. When asked whether this might lead to government regulation of algorithms, French President Emmanuel Macron, who announced the initiative with Prime Minister Jacinda Ardern, said the first step was to "cooperate" with tech companies

"The worst scenario is at the point of time we would be forced to regulate... it would be much more painful," Macron said.

The research initiative does not include some of the biggest social media companies like Meta, which owns Facebook or ByteDance, which owns TikTok. It will, however, involve the likes of Microsoft and Twitter.

Prime Minister Jacinda Ardern announced the effort with Macron after a Christchurch Call Summit on the sidelines of the United Nations General Assembly in New York.

It will fund a group called OpenMined to work with Twitter and Microsoft to understand how algorithms drive "content discovery". The research will protect the privacy of people's data and will help inform algorithm policy in the future.

The Christchurch Call was created in the aftermath of the March 15 terror attacks and has grown to include many tech companies and governments. The New York summit managed to attract the likes of European Commission President Ursula von der Leyen, Canadian Prime Minister Justin Trudeau and Spanish President Pedro Sánchez.

Representatives from Amazon, Google, Twitter and Microsoft also attended.

Ardern brushed aside criticism the research would be ineffective because it did not include the largest social media companies

"Once we are able to build that platform, that could be used across multiple platforms," Ardern said.

She said that many companies understood the "end point" of their algorithms, but not "how they got there".

She said that users would want to know how the content they received was being generated.

Macron expressed concern that tech companies were not spending enough time or money moderating their platforms, saying the money spent on moderation, particularly in languages like French, was "peanuts".

"All players are aware the current system is unsustainable."

Microsoft vice-chairman Brad Smith said the Christchurch Call had made "meaningful progress to stop the spread of violent extremist content online".

"The responsible use of AI, including explaining how algorithms recommend content to people on social media platforms is an important challenge we must address."