Easy, medium, hard: the missing variable in your AI cost math

Here's the problem with every LLM cost model in production today: it treats a complex multi-step reasoning request the same as a simple one-line classification, as long as they're both routed to the same model tier.

A "write a haiku about autumn" and "prove P ≠ NP" both hit Claude Sonnet. You pay the same per-token rate for both. One is trivially easy. The other is frontier-level difficult. The routing decision — model class selection — sees no difference between them.

That's the gap the Complexity Marker closes.

What Shipped

Trimio shipped a per-request complexity classifier (easy / medium / hard) that stamps every request with a difficulty score before routing decisions are made. The marker runs inline — no latency added — and feeds the Predictor's quality estimates with a second dimension: not just what kind of request it is, but how hard it is.

Why difficulty matters for routing

Least Cost Routing (LCR) works by matching each request to the cheapest model capable of handling it at the required quality level. The "capable" part is determined by the Predictor — which evaluates model quality scores against request traffic class to make the routing call.

Traffic class is one axis: a text-format request, a code request, a structured-output request. Each class has different quality thresholds — what passes for "good enough" in a bulk classification job differs from a production code review job.

But traffic class alone misses something important: complexity within the class. Two requests in the "text-format" class can have wildly different difficulty profiles:

"Summarize this paragraph" (easy)
"Synthesize findings across these 40 academic papers and identify the three most contradictory findings, then explain why they disagree" (hard)

Both are text-format. Both hit the same routing logic. But the hard request genuinely requires frontier-level capability — it needs long context reasoning, multi-step synthesis, and tolerance for ambiguity. The easy request would work fine on a much cheaper model.

The Predictor's quality score was calibrated against traffic class. It wasn't calibrated against difficulty within the class. That's the missing signal.

The Core Problem

LCR routes on two things: request type and model quality score. Neither captures request difficulty. A hard text-format job and an easy text-format job get routed identically — even though only one of them actually needs to hit the expensive model. Difficulty-aware routing fixes this.

How the complexity marker works

The marker is a three-class classifier — easy, medium, hard — running inline on every request. The model is a BGE-small ONNX head distilled from a brick-max teacher model (trained on `regolo/brick-complexity-2-max`). The distillation produces a compact, fast inference model that's purpose-built for difficulty classification.

The classifier runs before the routing decision — it sees the request, stamps it, and that stamp flows into both the request_logs and the Predictor's quality scoring pipeline.

The stamp includes four values:

complexity_easy — probability this request is easy
complexity_medium — probability this request is medium
complexity_hard — probability this request is hard
complexity_confidence — confidence in the classification

The Predictor receives the difficulty band as a Phase 8 anchor injection: a band-aware shift applied to the quality projection. A hard request in the "text-format" class gets a quality bar that's appropriate for a genuinely hard problem — not just a hard problem in that class, but a hard problem in absolute terms. The anchor shifts the Predictor's output so that easy-vs-hard within the class is now reflected in the routing call.

What this changes in practice

Before the complexity marker, a routing call looked like this:

Request: "Synthesize these 40 papers"
Traffic class: text-format
Predictor output: quality=0.82 → route to claude-sonnet-4.6

The Predictor said "Sonnet 4.6 will produce good quality for this text-format request" — which was true for easy text requests but not necessarily for a 40-paper synthesis. The complexity was invisible to the routing engine.

After the complexity marker, the same request gets a different routing call:

Request: "Synthesize these 40 papers"
Traffic class: text-format
Complexity: hard (0.91 confidence)
Predictor output (difficulty-injected): quality=0.63 → route to claude-opus-4.8

The Predictor now sees that this request is hard — and the quality bar that "good enough" sets for a hard problem is higher than for an easy one. The routing engine knows to send the hard job to a frontier model and route the easy jobs to cheaper alternatives.

The net effect: expensive models get used for requests that actually require them, and cheap models get used for everything else. The routing decision now reflects both what the request is AND how hard it is.

What Changes

Difficulty-aware routing means the Predictor now has two inputs: request type and request difficulty. A hard classification request gets routed to a better model than an easy one — even though they're both "classification." The complexity marker is the second axis the routing engine needed.

The financial implication

The CFO question is always the same: "why is our AI bill what it is?" The complexity marker gives Finance Reports a new dimension of answer.

Without difficulty labels, you can show: "X% of spend was on Sonnet, Y% on Opus, Z% on Gemini Flash." That's provider-level attribution.

With difficulty labels, you can show: "X% of spend was on hard-difficulty requests routed to Opus because they actually needed frontier capability. Y% was on easy-difficulty requests routed to budget models." And critically: "Z% of spend on hard requests could have been reduced by routing them to a cheaper model with lower error rate — here's what that would have saved."

That's a different CFO conversation. It's not "here's where the money went." It's "here's where the money could have gone, and here's how we fix it." The complexity marker turns the request_logs into a ROI diagnostic tool, not just a spend log.

What "ships dark" means and why it matters

The complexity marker is running today in production, stamping every request, feeding the Predictor — but nothing is routing on it yet. The marker is "dark": it's collecting data, learning the distribution of difficulty across Trimio's production traffic, and building the confidence baseline before any routing decisions use it.

This is the right engineering call. A routing change that affects which model processes which request is a high-stakes move. You want the classifier's accuracy validated against real production traffic before you let it affect routing. Running dark for a week or two before promotion gives the team:

A confidence baseline — what does "easy / medium / hard" distribution look like in real production traffic?
Error analysis — where does the classifier misfire? What's a false hard? What's a missed easy?
Routing simulation — run the difficulty scores against historical request logs and see what the routing decisions would have been, without actually changing them.

When the marker is promoted, it goes live as a routing input. The Predictor gets difficulty as a real-time signal. The LCR engine gets a second axis to optimize on. And the ROI story gets a new chapter.

Why Dark Shipping

The complexity marker runs in production without affecting routing so the team can validate accuracy against real traffic before letting it change which model handles which request. This is the correct order: collect data, measure error rate, simulate routing impact, then promote. Rushing the promotion before validation risks routing degradation.

What this means for the LCR engine

The current LCR architecture selects a model based on the Predictor's quality projection for a given traffic class. The complexity marker adds a difficulty axis to that projection.

Concretely: a "complex" classification request has a different quality threshold than an "easy" one. The Predictor's output for complex requests will shift toward higher-quality models. For easy requests, the bar drops — and cheaper models become viable candidates they weren't before.

The combination of traffic class + difficulty + quality score is the complete routing signal. Before this, LCR was operating on roughly two-thirds of the information it needed. The complexity marker closes the gap.

For engineering teams running Trimio: you don't need to do anything. The marker runs invisibly. When it promotes, your routing gets more accurate automatically. The financial reporting surfaces the new dimension. The LCR engine optimizes harder because it understands the problem space better.