Training AI models might not need enormous data centres
Eventually, models could be trained without any dedicated hardware at all
Once, the world’s richest men competed over yachts, jets and private islands. Now, the size-measuring contest of choice is clusters. Just 18 months ago, OpenAI trained GPT-4, its then state-of-the-art large language model (LLM), on a network of around 25,000 then state-of-the-art graphics processing units (GPUs) made by Nvidia. Now Elon Musk and Mark Zuckerberg, bosses of X and Meta respectively, are waving their chips in the air: Mr Musk says he has 100,000 GPUs in one data centre and plans to buy 200,000. Mr Zuckerberg says he’ll get 350,000.
Explore more
This article appeared in the Science & technology section of the print edition under the headline “I can do it with a distributed heart”
From the January 11th 2025 edition
Discover stories from this section and more in the list of contents
Explore the editionMore from Science & technology
Does melatonin work for jet lag?
It can help. But it depends where you’re going
How the Gulf’s rulers want to harness the power of science
A stronger R&D base, they hope, will transform their countries’ economies. Will their plan work?
Cancer vaccines are showing promise at last
Trials are under way against skin, brain and lung tumours
New firefighting tech is being trialled in Sardinia’s ancient forests
It could sniff out blazes long before they spread out of control
Can Jeff Bezos match Elon Musk in space?
After 25 years, Blue Origin finally heads to orbit, and hopes to become a contender in the private space race
Why some doctors are reassessing hypnosis
There is growing evidence that it can help with pain, depression and more