Since the debut of the AI assistant ChatGPT in November 2022, we’ve witnessed an AI arms race. Hundreds of billions have been spent on developing computational models and the high-performance computer chips and data centres required to run them.
The frenzy of spending by the likes of OpenAI, Microsoft, Google and Meta, was predicated on the concept of “neural scaling”, which suggests that AI capabilities increase predictably as systems grow in size and are trained on more data.
Those companies have hoovered up all of the information on the internet, often flouting copyright laws in the process, to train the large-language models that allow AI chatbots like Gemini and Perplexity to give you coherent responses to questions, and generate images, videos and entire computer programs in a matter of seconds.
Big Tech is now running out of quality sources of data to mine, forcing companies to turn to synthetic datasets, which are simulated to mimic real-world data. They are nowhere near as good as the real thing, introducing a new set of challenges and potential biases.
But the AI industry faces a bigger problem. Recent evidence indicates these scaling laws may be reaching a plateau. Simply making AI models bigger is no longer yielding proportional gains in capability. The whistle was most publicly blown on this trend in early December by none other than Google boss Sundar Pichai.
“I think the progress is going to get harder,” said Pichai. “The low-hanging fruit is gone … the hill is steeper. You’re definitely going to need deeper breakthroughs as we get to the next stage.”
That was refreshingly honest for a tech CEO whose company’s share price is closely linked to his ability to lead in the AI race. His claim was immediately disputed by Sam Altman, co-founder and CEO of arch-rival OpenAI, which created ChatGPT and was last year valued at more than US$160 billion.
There is no “AI wall” says Altman, who nevertheless delayed the launch of ChatGPT 5, citing constraints on computer capacity and a desire to focus on the numerous models and services the company has released.
A slowdown in the blistering pace of advances in AI is no bad thing given the fears expressed by credible experts, including Nobel Prize winner Geoffrey Hinton, the godfather of AI, that powerful AI poses an existential threat to humanity. It gives governments time to figure out how to regulate the technology and properly monitor its development. We can address the AI skills shortage, preparing the workforce for the disruptive change that is certainly coming. Security and privacy issues with the existing technology can be addressed before even more powerful models are unleashed.
AI companies must shift focus from raw power to efficiency if scaling hits its limits. This will lead to innovation in model compression, knowledge distillation and more targeted, domain-specific AI solutions.
A disturbing aspect of the rise of AI has been the scramble by tech companies to build data centres and secure access to electricity and water to power and cool them. The renewable energy capacity expected to come online in the next decade was supposed to facilitate the electrification of the global economy, not support a surge in ChatGPT and Copilot requests.
Slowdown or not, in 2025, AI will move from potential to production as it evolves from answering queries to actively assisting in complex tasks across various industries. The days of easy gains through scaling may be behind us, but this challenge presents an opportunity for more thoughtful, sustainable, and responsible AI development.
