Cerebras CS-4: 1,000 Tokens/Second — and Why Your Compression Layer Matters More

Cerebras announced the CS-4, their fourth-generation wafer-scale inference system. The numbers are straightforward: 1,000+ tokens per second on models exceeding 10 trillion parameters. 30× faster inference than GPU systems. 10× more throughput per watt than the CS-3.

The HN thread (324 points, 207 comments) focused on the speed. It should have focused on the billing model.

The prefix cache gap

From the HN thread, the most commercially important comment:

"They don't have a prefix cache like other providers, or at least don't have a discount in their billing structure. Each message charges for the whole context window. It's wildly more expensive for long multi-turn scenarios with lots of tool calls (coding). It's better for short few turn tasks."

This is not a Cerebras-specific observation. It is a routing decision. And it is exactly the kind of routing decision that should be automated, not manual.

When speed and cost pull in opposite directions

For a short, single-turn task — classification, quick Q&A, one-shot generation — Cerebras is the right answer. 1,000 tokens/second at Sol's post-cut pricing ($2.50/MTok input) is fast and cheap. The lack of prefix caching doesn't matter because there's no prefix to cache.

For a multi-turn coding session — 20 messages, 50K context, tool calls on every turn — Cerebras charges for the full 50K on every message. A GPU provider with prefix caching charges for the full context once and then caches the prefix at a fraction of the cost. The crossover point is typically 3–4 turns.

Why compression matters more on Cerebras than anywhere else

Here's the part the HN thread didn't connect. If every Cerebras message charges for the full context window, then every token you remove from that context is a direct, linear cost saving — not just on the first message, but on every subsequent message in the session.

A 40% context compression on a Cerebras-routed request is a 40% cost reduction on every turn. Not on the first turn. On every turn. Because there's no prefix cache absorbing the repeated portion at a discount, the compressed tokens are full-price tokens on every single message.

The arithmetic on a 20-turn coding session with 50K context:

• Without compression on Cerebras: 50K tokens × 20 turns × $2.50/MTok = $2.50 in input costs alone
• With 40% compression on Cerebras: 30K tokens × 20 turns × $2.50/MTok = $1.50 — a $1.00 saving from compression alone
• Same session on a cached GPU provider: 50K first turn + ~2K incremental per turn (prefix cached at 90% discount) × $3.00/MTok = ~$0.17 — cheaper, but only if your routing layer knows to pick it

The point is not that Cerebras is bad. The point is that the cost profile changes dramatically based on task shape, and the routing layer needs to understand the shape.

What a routing layer should do about this

Three things, all automatic:

1. Route by task shape. Short, few-turn tasks → Cerebras (fast, cheap, no cache needed). Long, multi-turn tasks → cached GPU providers (prefix cache absorbs the repeated context). The routing decision is based on conversation history, not on a static model preference.
2. Compress before routing. If the routing layer compresses context before sending to any provider, the savings are magnified on Cerebras specifically — because Cerebras charges full price for every token on every turn. Compression is not provider-specific, but its economic impact is provider-asymmetric.
3. Surface the savings. When the routing layer chooses a cached provider over Cerebras for a 20-turn session, it should show the customer why: "Routed to Anthropic instead of Cerebras for this session because prefix caching saves $X over 20 turns." When it routes to Cerebras for a single-turn task, it should show: "Routed to Cerebras for speed — 1,000 tok/s, no prefix cache needed for a single-turn task."

The CS-4 is an interim product

From the HN thread: the CS-4 is labeled "interim" before the WSE-4 family, which moves to a 3nm process with potentially 3D-stacked SRAM. If WSE-4 delivers on 3nm, Cerebras inference costs could fall further — expanding the gap between Cerebras speed and GPU speed, and potentially changing the crossover point where prefix caching becomes more economical.

But the billing model — no prefix cache, full-context charge per message — is a business decision, not a hardware constraint. It could change. If it does, the routing math changes. The routing layer needs to update automatically when provider pricing models shift, not wait for a human to notice.

Trimio's LCR engine routes based on task shape, conversation history, and provider-specific cost models — including Cerebras's full-context billing. Token compression runs before routing, magnifying savings on providers that charge full price per turn. Per-request routing evidence shows exactly why each model was selected. See how it works.