How electricity becomes intelligence
An AI data centre is a mill that turns electricity into words. Follow the power from the grid to the chip and back out as heat, and see why each part of the mill, in turn, became the thing everyone was short of, until the shortage reached the power grid itself.
trying to answer →What becomes scarce when intelligence becomes cheap?Where does value move when intelligence becomes commoditized?
pick a question; the machine answers it
Why can’t the richest companies on earth buy enough electricity?
A mill. Water turns the wheel and the wheel grinds flour. Here electricity flips billions of tiny switches, the switches do sums, and the sums pick the next word of a reply. Every watt goes in as power and comes out as heat. On DeepSeek’s published numbers, one kilowatt-hour, a kettle boiling for half an hour, writes roughly 4 million tokens, building and cooling included. This is the story of that mill, part by part.
$3,800
the electricity that $30,000 chip burns in five years. Power is the cheap part.
It is also the part nobody can get. This is how a problem inside a chip the size of a postage stamp walked out of the chip, out of the building and into the power grid, and how every step of that walk made somebody rich.
Why is a graphics chip running AI?
A graphics chip was built to draw video games: millions of pixels, each needing the same small sum at the same moment, so it has thousands of slow calculators instead of one fast one. In 2012 a small team trained an image-recognition network on two gaming cards and beat everything. A neural network is the same kind of work.
On record: AlexNet (Krizhevsky, Sutskever and Hinton, 2012) was trained on two NVIDIA GTX 580 graphics cards.
Chips used to get faster for free: each generation fitted more transistors into the same power. That stopped in the mid-2000s. Since then every jump in speed has been paid for in electricity. Then ChatGPT arrived, in November 2022, and everyone wanted the jump at once.
The chip maker, for as long as everyone’s software is written for its chips.
NVIDIA rises 24% in a day, adding about $184bn, on guidance of $11bn for the quarter.
Next: Why does a very expensive chip spend its time waiting?
Why does a very expensive chip spend its time waiting?
Picture a chef who chops faster than anyone can carry ingredients to the counter. The chip got faster at arithmetic than memory could feed it numbers. Every word an AI writes means reading the whole model out of memory again, to do a little arithmetic with it. So the chip mostly waits.
So memory moved closer: stacked into towers, stood millimetres from the chip, joined by thousands of very short wires instead of a few long ones. More than ten times the bandwidth. It had been on sale since 2015, and almost nobody had wanted it.
The same problem was attacked in the code, by papers that each found a way to make the chip wait less. Why each one mattered →
- KV cache Why does a long conversation get slower and cost more than a short one?
- FlashAttention Why is attention slow when the GPU is barely doing any arithmetic?
- PagedAttention Why does a GPU with free memory still refuse new requests?
- Mamba Why does a model reread the whole conversation instead of just remembering it?
Three companies in the world make this memory, and at first only one of them could make it well enough.
TrendForce reports NVIDIA's HBM3 came at first from SK Hynix alone.
Next: Why is an AI chip really a sandwich?
Why is an AI chip really a sandwich?
A single chip can only be so big: the machines that print them expose one fixed-size field at a time. So the industry started gluing several chips and their memory towers onto one slab of silicon wiring and calling the sandwich a chip. That gluing step used to be the cheap afterthought at the end of the line.
52–78 weeks
the wait for that gluing step, with lines sold out into 2027
The chipmaker that owns most of it, and behind it an insulating film made by Ajinomoto, a company better known for seasoning.
TSMC's chairman says the shortage is of its CoWoS packaging capacity, not of AI chips, and will last about 18 months.
Next: Why can’t air cool it any more?
Why can’t air cool it any more?
A 300-watt chip with a fan on it was solved for twenty years. Now it is 1,200 watts through the same few square centimetres, and the heat has to climb out through a stack of layers, pulled apart here so you can see them. Each layer resists a little. Four times the heat makes every little resistance four times worse.
Air cannot carry that away, so water came back, as it once did in mainframes: a cold plate on every chip, pipes to every rack. It brings plumbing, leaks, and a building that has to have water in the first place.
The makers of cold plates and cooling units, and whoever owns a building that already has water.
NVIDIA announces GB200 NVL72, a rack that ships liquid-cooled.
Next: Why did the whole cabinet become the computer?
Why did the whole cabinet become the computer?
For forty years you bought computers one server at a time, because one box was big enough to hold a useful machine.
Now seventy-two chips have to act as one, talking all the time, and that only works if they sit centimetres apart. So the thing you buy is a whole cabinet, drawing over 100 kilowatts where one used to draw about 10.
The companies that build these cabinets, but only while what they sell is scarce. Selling more than ever is not the same thing.
Supermicro falls 33% in a day after its auditor resigns.
Next: Why are data centres now built around electricity?
Why are data centres now built around electricity?
At 10 kilowatts a cabinet, the power a building lost converting electricity was a rounding error. At 120, the same few percent is a room of heavy copper and transformers the building was never drawn with.
So a new data centre starts from one number, how much power can I get, and everything else is drawn around it. Two to four years from land to first power, at roughly ten million dollars a megawatt.
Whoever already holds land with power, permits and water. Any one is easy to find; all four at once is rare.
Next: Why can’t you just buy more power?
Why can’t you just buy more power?
When a big data centre needed 30 megawatts, the utility had it spare. Nobody queued.
4–7 years
to connect a big new load to the grid. A new AI model comes out every few months.
Utilities, power plants, and whoever already holds the right to connect.
Constellation rises 22% on a 20-year, 835 MW deal to restart a Three Mile Island reactor for Microsoft.
Next: So where does it go next?
So where does it go next?
Nothing here was ever fixed. It only moved: chip, memory, package, heat, cabinet, building, grid. Each fix made the next thing scarce, and whoever held the scarce thing got paid. The next constraint is already outside the fence: turbines, transmission lines, and a public argument about who the power is for.
DeepSeek: NVIDIA falls 17%, $589bn, the largest one-day loss in US market history. Vistra falls 28%; Constellation and GE Vernova more than 20%.
that was one question. ask another?