009How much compute does AI demand need by 2027?

Four things set what a token costs

The model. Compute per token rises with the parameters a model uses for each token more than with its total size, though total size and the length of the conversation matter too. On one B200, gpt-oss-120b, which uses 5.1B parameters per token, makes 3.8B tokens a day. DeepSeek-R1, at 37B, makes 0.40B. MiniMax-M2.5 uses a quarter of GLM-5’s active parameters and makes 3.9× its tokens. The rule is rough: Llama 3.3 70B, which uses all 70B of its parameters for every token, makes more than DeepSeek-R1. We size every model by its active parameters.

The hardware. Serving DeepSeek V4 Pro to coding agents at 100 tokens/s per user, a megawatt of GB300 NVL72 racks serves 13× the tokens of a megawatt of H200s running at 8-bit precision, and 4.1× those of B200s. Vera Rubin, now shipping, doubles GB300 again. Racks of 72 GPUs win on large models because they spread the model’s experts and its memory of the conversation across many chips. At full load, a gigawatt of GB300s would get through about 100 days of OpenRouter’s October volume each day, if every token were a DeepSeek V4 Pro agent token.

The kind of token. Output is 2.4% of OpenRouter’s tokens but about 34% of the GPU time behind them. New input is 19% of the tokens and 55% of the time. Cached input, the context agents send again at every step, is 79% of the tokens and about 11% of the time. These weights follow the labs’ own pricing: output costs about five times new input, and a cache read a tenth to a twentieth of it; we use a twentieth.

Speed and memory. Serving each user faster means smaller batches. DeepSeek-R1 on a B200 makes 6,799 tokens/s per GPU at 30 tokens/s per user and 2,685 at 100. A model that doesn’t fit on one server is worse: at 50 tokens/s per user, DeepSeek-R1’s best H100 setup made 74 tokens/s per GPU, against 4,610 on a B200.

OpenRouter in GPUs

At SemiAnalysis’s measured throughput and 60% utilization, October’s 23.8T tokens a day would need about 20 MW of GB300 racks, about 9,300 GPUs. On H200s they would need 250 MW. On a mix of hardware like the industry’s installed base today, part Hopper, part Blackwell, part TPU and other chips, they would need about 60 MW. Every megawatt here is all-in: chips, servers, networking and cooling. Real deployments run older software than a tuned benchmark, which pushes the need up; how closely the benchmark’s traffic matches OpenRouter’s moves it either way (see Method).

The compute isn’t where the tokens are. Open models that use 20B parameters or fewer per token, led by DeepSeek V4.1 Flash with 21% of all tokens, are 50% of the tokens, 17% of the compute and 4% of the dollars. Closed models above that size, among them Claude Opus 5.5 and Sonnet 5.5, GPT-6 Sol and Astra and Gemini 3.8 Flash, are 12% of the tokens, 64% of the dollars and, by our price-based estimate, 49% of the compute. Opus 5.5 alone is 3% of the tokens and about 18% of the compute. Those compute shares rest on what we assume about the labs’ margins: at gross margins of 45–75% instead of 60%, the larger closed models are 38–56% of the compute.

OpenRouter in 2027

In the base case OpenRouter handles 513T tokens a day by December 2027, 22× October’s, while spend grows only 4.2×, so the average price of a token falls 5.1×. Holding each model’s price fixed, as we do, that requires cheaper models to keep taking share: in the mix that reproduces pack 007’s numbers, DeepSeek V4.1 Flash’s tokens grow 48× and Claude Opus 5.5’s 1.5×. Because the cheap tokens need little compute each, compute grows 10×, from 20 MW of GB300s to 195 MW, about 84,000 more GPUs. On today’s mix of hardware it would be 600 MW. If prices fall model by model instead, as GPT-5.6 Luna’s did when OpenAI cut them 80% this summer, compute grows closer to tokens.

The low path needs 4.1× today’s compute (80 MW of GB300s) and the high path 23× (460 MW). Even the high case is under half a gigawatt of GB300s. OpenRouter on its own is a rounding error in a year that adds tens of gigawatts. What matters is what it says about everyone else.

From OpenRouter to the market

OpenRouter is a small slice. On matching dates its tokens were about 3% of Google’s across all its products, 1.3% of a national total for China that TechNode reported for June without a source or method, and 0.6–2.2% of published estimates for the whole world. Its share of the compute is smaller still, about 0.4% of the chips doing inference by our estimate. Partly that is because its tokens are cheap: half come from small open models that coding agents call over and over. Partly it is because benchmarks flatter real deployments: at benchmark throughput, today’s inference chips could serve about 6.2 quadrillion OpenRouter-like tokens a day, 6.3× the highest estimate for the world.

So we scale up from the hardware, not the tokens, and use the benchmarks only to compare one kind of hardware with another. Epoch AI estimates 31.6M H100-equivalents of AI chips operating by September: chips delivered through June, allowing about three months to install. At about 1.25 kW each all-in, that is 39.5 GW. If 40% of it serves inference, that is 16 GW. If the market’s inference demand grows the way OpenRouter’s compute does in the base case, 10× by December 2027, it needs about 160 GW of hardware like today’s. Built as GB300s, which serve about 3.1× as many tokens per MW as today’s average, the increase is about 47 GW, or 22 million GPUs. Built as Vera Rubin, it is about 22 GW. Almost all of it falls in 2027: 43 GW of GB300s between December 2026 and December 2027.

That takes the premise at face value, and the premise is aggressive. At full pace the market’s inference compute grows about 7.2× a year, roughly what Google’s tokens did in the year to May; over the same year OpenRouter’s tokens grew 12.5×. The installed base of AI chips grew 3.0× in 2025. At half OpenRouter’s growth rate, about 2.7× a year, close to the chip stock’s growth, market demand rises 3.2× by December 2027 and needs 11 GW of new GB300s. Whether the answer is closer to 11 GW or 47 GW depends on whether the rest of the market speeds up toward OpenRouter’s pace.

Three inputs we can’t pin down move the 47 GW: the share of today’s chips doing inference (30–50% gives 35–58 GW), how efficient today’s fleet is (0.75–1.3× our estimate gives 35–60 GW) and how many chips are installed (Epoch’s range, roughly 1.5× either way, gives 31–70 GW). Moving all three together gives 17–113 GW.

What 2026 added

Epoch AI’s chip-sales data counts 11.3M H100-equivalents sold in the first half of 2026, against 13.6M in all of 2025. If the second half runs 10–40% above the first, 2026 adds 24–27M, more than the 20.3M delivered in all the years through 2025. Those chips are rated at 12–14 GW. By Epoch’s rule of thumb a data center needs about twice its chips’ rated power once servers, networking and cooling are counted, so 2026’s chips need 25–28 GW all-in. NVIDIA’s data-center revenue was $164B in the first half of its fiscal 2027, against $194B in all of fiscal 2026, and Vera Rubin began production shipments in August. NVIDIA expects supply to stay the bottleneck at least through January 2028.

Not all of it serves inference. At 40%, 2026 gives inference about 10–11 GW, on a mix of hardware that serves fewer tokens per MW than GB300 racks. Demand at OpenRouter’s pace would need about 4.1× that in 2027 alone if it is built as GB300s, or 1.9× as Vera Rubin. At half the pace, 9.7 GW of GB300s in 2027 is about the inference share of a 2026-sized year, and more if built from 2026’s mix of chips. This pack is about demand; a later one will look at how much capacity 2027 can actually deliver.

What would change the numbers

The pace. It is the biggest lever. From a quarter of OpenRouter’s growth rate to all of it, the need by December 2027 runs from 4.0 GW to 47 GW of GB300s.

The mix. Our compute grows less than tokens because, at fixed prices, cheap models keep taking share. If instead the mix froze and every model’s price fell, compute would grow with tokens, 22×, and the market would need about 106 GW. Two other choices matter less. If the larger closed models’ tokens grew 3.5×, as fast as OpenAI’s and Anthropic’s revenue run-rates in pack 008, the market would need 45 GW. If free tokens stayed flat instead of growing with the rest, 42 GW.

How big the closed models are. Nobody discloses them, so we infer their size from price, assuming the labs keep 60% of it as gross margin. At 45% the models are bigger, take more of today’s compute and grow more slowly: 41 GW. At 75%, 55 GW.

How compute scales with size. We make compute per token proportional to active parameters. Across InferenceX’s six B200 results it rises more slowly, roughly with active parameters to the power 0.89. On that basis small models cost relatively more, the shift toward them saves less, and the market would need 53 GW.

Software. Benchmarks move every month. InferenceX has measured gains of 1.5–2× from software alone on the same chips within weeks, on DeepSeek-R1 and on DeepSeek V4 Pro. Each doubling roughly halves the need; we hold today’s software fixed.

Speed. We measure at 100 tokens/s per user. Serving agents faster costs throughput steeply: on DeepSeek V4 Pro, a GB300 rack serves about a fifth as many tokens per MW at 150 tokens/s per user as at 100, while Vera Rubin holds up better. Faster agents would raise every figure here and widen Rubin’s lead.

Output. Agents write about 1% of their tokens. An output token costs about five times a new input token, so agents that write more would raise compute faster than tokens.

What to watch

Google’s next token count. Its third-quarter results, due in late October, are the cleanest read on whether the rest of the market is growing near OpenRouter’s pace.

Rubin’s share of 2027 deliveries. Each GW of Vera Rubin does the work of about two of GB300s, so the mix of what ships in 2027 roughly halves or doubles the gigawatts needed.

Cache rates. OpenRouter’s agent traffic went from about 60% cached in January to nearly 90% in September, which kept compute growth below token growth. It can’t rise much further, so from here more of token growth turns into compute growth.