Cerebras disaggregated inference: AMD GPUs for prefill, CS-3 for decode, 5x tokens per megawatt
Cerebras says GPUs now read the prompt while its wafer-scale systems write the answer, raising throughput per megawatt fivefold on one endpoint, by its own measurement.
· 6 min read
Written by AI from the sources linked below. How we use AI
On this page
Cerebras says it has put disaggregated inference into production: GPUs handle the prompt (prefill) and Cerebras wafer-scale systems generate the answer (decode). The company reports five times more tokens per megawatt than a Cerebras-only setup, with no loss in generation speed. The claim is Cerebras's own, from one endpoint on one day, but the design matters because it shows where AI inference hardware is heading: different chips for different phases of the same request.
Key points
- Cerebras announced on 17 November 2026 that disaggregated inference is in production on its Inference Cloud, with GPUs for prefill and Cerebras systems for decode.
- The company says throughput per megawatt is 5x a CS-3-only configuration, measured on its shared GPT-OSS-120B endpoint on 14 November, with no loss in token generation speed.
- The production deployment pairs AMD Instinct GPUs (prefill) with CS-3 systems (decode); Cerebras plans hundreds of megawatts of this by the end of 2027.
- Early-access partners named by Cerebras are Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute.
- Cerebras did not publish prices, absolute tokens per second or latency for the new setup, and the speed comparisons come from its own or third-party testing.
What happened
In a blog post dated 17 November 2026, Cerebras described how it now serves some inference traffic. A request has two phases. Prefill reads the whole prompt in one parallel pass and builds the intermediate state, including the key-value (KV) cache. Decode then produces output tokens one at a time, and for every token it has to read the model's weights from memory again. Prefill is limited mostly by raw compute; decode is limited mostly by memory bandwidth.
Cerebras's chips keep model weights in on-chip SRAM. By the company's figures, a chip has tens of gigabytes of SRAM and tens of petabytes per second of memory bandwidth, which it puts at 350 times the SRAM capacity and 5,400 times the memory bandwidth of a typical Nvidia GB300 GPU. That suits decode. It is less of a match for prefill, which does not need that bandwidth but does take up the chip's time. So Cerebras moved prefill elsewhere. On its Inference Cloud, AMD Instinct GPUs now do the prompt processing and CS-3 systems do the decoding, with the KV cache passed between them.
Cerebras says the result, measured on its GPT-OSS-120B shared endpoint on 14 November, is five times the throughput per megawatt of a configuration where CS-3 systems do both jobs, with token generation speed unchanged. The throughput was normalised to a 1x baseline of the aggregated setup. Cerebras plans to roll this out across hundreds of megawatts in its own cloud and partner deployments by the end of 2027.
It is also offering the design to infrastructure partners. One option is an end-to-end stack: a framework that routes requests and tracks pool capacity and health, a prefill engine that supports open-source engines such as vLLM or SGLang, the Cerebras decode engine, and a component called KV Portal that moves model state over RDMA. The other is a decode-only SDK for partners who bring their own framework and prefill engine. Cerebras names Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute as early-access partners.
Background
The idea of splitting the phases is not new. Cerebras began a blog series on it on 1 October 2026, which Unite.AI summarised at the time. That post reported the same 5x capacity gain in early tests on an unchanged number of Cerebras systems, with AWS Trainium and AMD Helios Instinct systems drawn as prefill options. Unite.AI notes that the 5x figure was the company's own and came with no details of workload, batch settings or latency.
The earlier partner announcement came from AWS. On 13 March 2026, AWS and Cerebras said they would offer inference through Amazon Bedrock using what they called inference disaggregation: AWS Trainium for prefill, Cerebras CS-3 for decode, connected by AWS's Elastic Fabric Adapter networking on the Nitro System. AWS said it would be the first cloud provider with Cerebras's disaggregated solution and expected it to launch within a couple of months. AWS described the result as "an order of magnitude faster" than what is available today, a forward-looking claim and not a measurement.
The 17 November post names AWS only as an early-access partner and says the production deployment on Cerebras's own cloud uses AMD GPUs. It does not say whether the Trainium and Bedrock service has launched, so we cannot confirm that from the sources we could read.
How it works
The reason to split the phases is that the two have opposite needs. Cerebras's October post explains it with arithmetic intensity, the number of floating-point operations per byte moved. Prefill touches many tokens at once, so it does a lot of arithmetic per byte of weights read. Decode handles one token per step per request and mostly waits for memory. Running both on the same hardware means one of the two resources sits partly idle.
Disaggregation has costs. The KV cache has to cross a network between the pools, which adds latency and engineering work. If one pool is sized wrong for the traffic, the other sits idle. Cerebras's own post argues the approach pays off at scale, and that it beats other splits it considered (draft and target models, attention and feed-forward layers, or serving everything on one system) for highly interactive workloads.
The October post also gives context for why Cerebras leads with decode speed. Using Artificial Analysis data from 10 September 2026 for GPT-OSS-120B at high reasoning with 10,000 input tokens, it listed output speeds that put Cerebras well ahead of other providers:
This is a single model and one workload setting, and it comes from a chart in the vendor's own post, so treat it as the vendor's framing of an independent dataset.
What it means
For operators, the pitch is efficiency per unit of power. Cerebras cites Goldman Sachs estimates that inference demand grows 24 times while data-centre capacity grows 2.2 times by 2030. Whatever the exact figures, power is the constraint the post is written around, which is why the headline metric is tokens per megawatt and not tokens per dollar.
For developers using fast-inference APIs, the immediate effect is small: Cerebras says speed is unchanged, and no new prices were announced. The longer-term effect is more choice. A design where prefill can run on whichever GPU a partner already has lowers the barrier for clouds that own AMD or AWS silicon and want Cerebras-class decode speed.
For rivals, it is a sign that heterogeneous serving is becoming normal. Cerebras is positioning wafer-scale decode as a component that attaches to other vendors' accelerators and not only as a standalone product.
What we don't know yet
- Prices. Cerebras did not disclose pricing for the disaggregated service.
- Absolute speed and latency. The 5x figure is a ratio against Cerebras's own previous setup, not a comparison with other providers, and the post gives no tokens-per-second or time-to-first-token numbers for the new configuration.
- Independence. The measurement is from one endpoint over a window of a few hours on 14 November. We found no independent benchmark of it.
- Whether the AWS Trainium and Bedrock offering announced in March is live.
- How the gain holds up for other models, longer prompts or mixed traffic.
What to watch next
Look for independent measurements from benchmarking groups, the first partner deployments outside Cerebras's own cloud, pricing, and any follow-up posts in Cerebras's series, which says it will cover the economics and the software stack later. Also watch whether other chip makers with large on-chip memory pitch themselves as decode accelerators in the same way.
Sources
Written by
The official account of the people who run audiotale.net: announcements, conversation starters and the newsroom's drafts. Posts written with AI say so. Every other account here is a real person.
Read next
Cerebras explains how Kimi K3 fits on 59 wafers: 93 layers, 44 GB each
Cerebras's worked example shows how a roughly 3-trillion-parameter open model is split across wafers with all weights kept in on-chip memory; we recheck the sums.
audiotale.net team7 min read
OpenAI models cut AMD GPU design PPA by up to 14%, Jalapeño area 13%
OpenAI says its models improved heavily optimised AMD GPU circuits, with some changes already in a design sent to manufacturing, and is building GPT-Synopsys with Synopsys.
audiotale.net team6 min read

BIS remote-access chip rule: licences for 5 Asian countries, 12M TPP cap
The US Commerce Department's 16 November rule requires licences for advanced AI chips to Indonesia, Malaysia, Singapore, Thailand and Brazil, and for data centres serving Chinese-headquartered customers.
audiotale.net team7 min read