014How many tokens can a GPU make?
DeepSeek open-sourced DeepInfer, the engine its own API runs on. Its benchmarks show a Blackwell GPU producing 26,300 output tokens a second at about 103 tokens a second per user, and DeepSeek says it earns $38 an hour per GPU at its off-peak prices, about three times its 2025 figure. Only Blackwell data-center GPUs and Huawei’s Ascend 950 can run it. Three days later vLLM published results within 6% of it, so the yield is now available to anyone with a Blackwell rack.
Output per GPU falls as each user gets faster
DeepInfer decode on Blackwell at a 131,072-token context, as DeepSeek benchmarked it: output tokens per second per GPU against tokens per second per user, one dot per batch size. EP64 spreads the model’s experts over 64 GPUs in one NVLink domain, which takes a 72-GPU rack; EP8 over one 8-GPU box. Benchmark figures, before DeepSeek’s 80% haircut for live traffic.
What DeepSeek released
On October 20 DeepSeek put DeepInfer, the engine behind its own API, on GitHub under the Apache 2.0 license. It serves one model, V4.1-Flash, and runs only on NVIDIA’s Blackwell data-center GPUs (B200, B300 and their 72-GPU racks) and Huawei’s Ascend 950. Hopper GPUs such as the H100 and H200 fail its startup check, and AMD isn’t a build target. It is the offline core: the cache store, request routing and prefill/decode split a live service needs are left out, and the smallest efficient deployment is 64 prefill plus 64 decode GPUs. DeepSeek says the open build is within 5% of its internal one and that live traffic reaches about 80% of these benchmarks.
Does it change GPU usage?
Not for DeepSeek, which has run this stack since V4.1-Flash launched in September. For everyone else it set the bar, and the bar was cleared fast. On the dashboard we built on release day, DeepInfer’s curves implied about 2.0× the tokens per GPU vLLM had published on agent traffic a few days earlier (830K a second). On October 23 vLLM, the most widely used open-source engine, published decode results within 6% of DeepInfer’s across its batch sizes, using DeepSeek’s kernels, and 954K tokens a second per GPU on the same agent workload on a GB300 NVL72 rack, with the online serving DeepInfer lacks. That is 15% more than its October 17 figure, with a P90 speed of 196 tokens a second per user against a 100 target then. Anyone hosting V4.1-Flash on a Blackwell rack can now get close to DeepSeek’s own yield with an open engine, which cuts the GPUs a host needs for the same traffic.
Does it change token yields?
Against 2025, by a lot. At its production point DeepSeek gets 6.3× the output per decode GPU of its V3 fleet on H800s, at 11.5× the speed per user, and $38 of revenue per GPU-hour against $12.91, even after cutting prices by about three-quarters. Most of that arrived with the model: V4.1-Flash stores 890 bytes of cache per token, against 48,068 for V3.2. The engine is how DeepSeek cashes it in. But the yield depends on the speed sold: 26,300 tokens a second per GPU at 103 per user, falling to 26,300 at 103.
Do GPU types change differently?
Yes, and topology matters more than the chip. Decode wants 64 to 128 GPUs in one NVLink domain, which only GB200 and GB300 NVL72 racks offer. An 8-GPU B200 or B300 box keeps up at relaxed speeds and tops out at 239 tokens a second per user. Hopper and AMD gain nothing directly. Ascend 950 is a first-class target, which matters more for China’s chip mix than for anyone’s throughput.
Cost per million output tokens at market rental
Typical rental per GPU-hour on October 20 ÷ output per GPU-hour, after the 80% live-traffic factor. GB200 NVL72 on the EP64 curve at $10.58 an hour; HGX B200 on the EP8 curve at $8.02, scaled to its 90% peak rate. The dashed line is DeepSeek’s off-peak output price. GPU time only: prefill, cache storage and the rest of the stack are extra.
The 8-GPU box is the cheaper way to buy output up to about 130 tokens a second per user; past that the rack wins, by a widening margin. The box costs more per output token than DeepSeek charges off-peak beyond about 225; the rack holds out to about 310. At equal speed the rack makes 1.1× the box’s output per GPU at 105 tokens a second per user and 6.4× at 235.
What DeepInfer changes, chip by chip
Per-GPU specs from vendor sheets and coverage, HBM as vendors list it. Scale-up domain is the most GPUs one interconnect domain holds. Rental is the typical on-demand price per GPU-hour on October 20. “What changes” is our assessment.
| GPU | DeepInfer | FP4 PFLOPS | HBM GB | Scale-up | Rent / hr | What changes |
|---|---|---|---|---|---|---|
| GB200 NVL72NVIDIA | Runs | 10 | 186 | 72 | $10.58 | Full effect. The setup DeepSeek benchmarks: EP64 decode inside one 72-GPU NVLink domain, Mega MoE kernels, native FP4 weights. |
| GB300 NVL72NVIDIA | Runs | 15 | 279 | 72 | $14.59 | Full effect expected; FlashMLA and DeepGEMM build for SM 10.3. Extra HBM matters less now that KV no longer binds; extra FP4 compute helps prefill. Rents for about 38% more than GB200 per GPU-hour. |
| HGX B200NVIDIA | Runs | 9 | 180 | 8 | $8.02 | Partial. Runs DeepInfer, but the NVLink island caps decode at EP8: close to parity at relaxed speeds, falls behind fast. DeepSeek uses 8-GPU boxes for prefill. |
| HGX B300NVIDIA | Runs | 13.5 | 270 | 8 | $9.99 | Partial, as B200. More HBM and FP4 compute don't lift the EP8 ceiling on fast decode; well suited to prefill. |
| H200NVIDIA | No | — | 141 | 8 | $5.33 | No direct effect. The engine's startup check rejects anything but SM 10.x and Hopper has no FP4 tensor cores. Gains arrive only as vLLM and SGLang port the ideas. |
| H100NVIDIA | No | — | 80 | 8 | $4.17 | No direct effect, as H200, with less HBM. |
| H800NVIDIA | No | — | 80 | 8 | — | No direct effect. The chip DeepSeek served V3 and R1 on in 2025. |
| H20NVIDIA | No | — | 96 | 8 | — | No direct effect; also the weakest compute in this table. |
| MI355XAMD | No | 10.07 | 288 | 8 | $5.60 | No direct effect. FP4-capable, but DeepInfer's kernels are CUDA or Ascend only; runs V4.1-Flash through vLLM, SGLang or ATOM. |
| MI300XAMD | No | — | 192 | 8 | $3.65 | No direct effect; no FP4 matrix units, so FP4 experts are dequantized in software. |
| Ascend 950Huawei | Runs | 2 | 144 | 8,192 | — | Strategic. First-class target (DeepGEMM-Ascend, DeepEP-Ascend, TileLang). DeepSeek and Huawei built a 128-chip Ascend 950 supernode that fits the decode design; Huawei's SuperPoD scales to 8,192 chips. Peak FP4 per chip is a fifth of GB200's. |
Most tokens are cache reads; most revenue isn’t
How $38 per GPU-hour adds up at V4.1-Flash off-peak prices: one prefill GPU per decode GPU, as DeepSeek runs them, both at the production point and the 80% factor. We solve for the share of input that hits the cache so the pair averages DeepSeek’s figure: 95%, close to the 96% that agent benchmarks measure.
At those rates about 2,700 GPUs would carry the 155T tokens a day DeepSeek reports and bill about $0.9B a year off-peak, or $1.1B counting peak hours at double. Reported annualized revenue for all of DeepSeek was about $1.0B at the start of October, so the figures hang together. vLLM’s independent $38.96 on a GB300 NVL72 points the same way.
Memory no longer limits the batch
How many 131,072-token sessions fit in 100 GB of HBM, by generation, log scale. At DeepInfer’s largest decode batch, 256 such sessions take about 30 GB per GPU, so bandwidth and communication now set decode speed, not memory.
Then and now
DeepSeek’s February 2025 fleet averages for V3 and R1 on H800s against V4.1-Flash on DeepInfer at its production point, after the 80% factor. Different chips and price decks; V3 revenue is DeepSeek’s theoretical figure at R1 list prices.
| Measure | V3 / R1 on H800 · Feb 2025 | V4.1-Flash on DeepInfer · Oct 2026 | Change |
|---|---|---|---|
| Output tokens/s per decode GPU | 1,850 | 11,573 | 6.3× |
| Per-user output speed, tokens/s | 21 | 241 | 11.5× |
| GPU-hours per billion output tokens | 150 | 24 | −84% |
| Revenue per GPU-hour | $12.91 | $38.00 | 2.9× |
| Output, per 1M tokens | $2.19 | $0.60 | −73% |
| Uncached input, per 1M tokens | $0.55 | $0.15 | −73% |
| Cached input, per 1M tokens | $0.140 | $0.003 | −98% |
Where the gain comes from
The model. V4.1-Flash keeps 890 bytes of cache per token, 3.9× less than V4-Flash, and activates 8B parameters when reading and 16B when writing. Its launch prices show where costs fell: cached input 57% cheaper than V4-Flash, uncached input 32%, output only 9%.
The engine. Wide expert parallelism is most of it. Bounded replay rebuilds only the last 140 tokens of sliding-window state, which DeepSeek says nearly halves prefill work. Confidence-scheduled drafting verifies fewer guessed tokens per step and adds 5% throughput at the production batch, 11% at the largest.
The hardware. The model’s expert weights are in FP4, which Blackwell, AMD’s MI355X and Ascend 950 multiply natively and Hopper does not. Decode wants a big NVLink domain, which is why the rack beats the box.
What it means
For GPU economics. A GB200 rents for about $10.58 an hour and, served this way, earns $38 at DeepSeek’s off-peak prices. Serving open models on Blackwell racks is profitable at market GPU prices, so long as the traffic is cache-heavy agent work, and vLLM now makes that available to every host. Third-party hosts charge about DeepSeek’s peak rate, so there is room for their prices to fall toward its off-peak ones (pack 006).
For older chips. Hopper rents for less but gets none of this directly, and it can’t do the FP4 math. The yield gap between a Blackwell rack and an H100 widens with each release like this one, which should weigh on Hopper rental prices over time.
For China. DeepSeek built Ascend 950 support in from the start and is reported to be planning a 160,000-chip Huawei data center. DeepSeek hasn’t published any Ascend throughput, which is the number to watch.
What this is anchored on
Decode and prefill points are read off DeepSeek’s benchmark charts; rental prices and DeepInfer figures as of October 20, vLLM’s as of October 23.
| Item | Data point | Source |
|---|---|---|
| DeepInfer | Engine, supported hardware, decode and prefill benchmark charts, $38 per GPU-hour, 155T tokens a day | DeepInfer repository |
| DeepInfer | Production concurrency, draft verification table, balance factor, KV storage, minimum deployment | DeepInfer deployment Q&A |
| V4.1-Flash | Parameters, 890 bytes of KV cache per token, KV series by generation | DeepSeek-V4.1-Flash technical report |
| V4.1-Flash | KV-cache series by generation | Context Studios |
| Prices | V4-Flash and V4.1-Flash off-peak and peak rates; peak hours a week | Apidog |
| vLLM | Within 6% of DeepInfer on decode; 954K tokens/s per GPU on AgentX, GB300 NVL72; $38.96 per GPU-hour (Oct 23) | vLLM blog |
| vLLM | 830K tokens/s per GPU on AgentX at 100 tok/s per user (Oct 17) | AICoder |
| vLLM | 5× high-throughput gain on V4.1-Flash since day 0 | vLLM blog |
| AgentX | Median input and output tokens, cache-hit rate | AgentX workload |
| AgentPerf | Tokens/s per MW by system on V4.1-Flash | Artificial Analysis |
| AgentPerf | MI355X, B200 and B300 results | CryptoBriefing |
| Rental | Typical on-demand price per GPU-hour, Oct 20 | Savrn AI index |
| Ascend 950 | Specs | TechRadar |
| Ascend | DeepSeek’s Ascend tooling; planned Huawei data center; $1B run-rate | The Next Web |
| DeepInfer | Release coverage | MarkTechPost |
| DeepInfer | Release coverage | The Next Web |
| V3 / R1, 2025 | DeepSeek’s Feb 2025 inference-system figures on H800s | Analytics Vidhya |
Method
Reading the charts. DeepSeek’s decode chart defines output per GPU as per-user speed × requests per GPU, and our readings meet that within 0.5%. Decode output assumes 75% acceptance on 3 draft tokens, as DeepSeek’s chart does; only the step rate is measured. The EP64 curve ran inside one NVLink domain, so on a GB200 or GB300 NVL72 rack; DeepSeek doesn’t say which.
The production point is DeepSeek’s draft-verification table at 60 sequences per GPU, which it says is where production runs. “After the live-traffic factor” multiplies by 80%, DeepSeek’s ratio of live traffic to its perfectly balanced benchmarks.
Costs use typical on-demand rental per GPU-hour, not DeepSeek’s own costs, which it doesn’t disclose. HGX B200 output is the EP8 curve scaled by DeepInfer’s own peak-rate table (2,250 against 2,500 TFLOPS).
The revenue check assumes every GPU runs at the production point around the clock, with prefill at 32,768-token steps. The comparison with vLLM’s earlier result priced one agent request (131,072-token context, 4,096 uncached, 444 output tokens) in DeepInfer GPU-seconds; it set an offline benchmark against a live replay, so it was indicative, and vLLM’s like-for-like decode test on October 23 supersedes it.
What DeepSeek doesn’t disclose: which Blackwell part ran the benchmarks, whether the cluster behind the daily token figure is NVIDIA, Ascend or both, fleet size, the real cache-hit share, and any Ascend 950 throughput.