What Is Model Routing?
Model routing is the practice of sending each AI task to the model best suited to it, usually the cheapest one that can do the job reliably, and escalating to a larger, more expensive model only when the task needs deeper reasoning.
Also known as:
How Model Routing Works
A router looks at an incoming task and decides where it should go. The decision can be based on simple rules, a classifier or the result of a first, cheaper attempt.
- Rule-based routing. Known task types always go to a set model, such as a small model for classification and a large one for code changes.
- Classifier routing. A lightweight model or heuristic predicts how hard the task is and picks a model to match.
- Cascading. A cheap model tries first, and the task escalates only if it is uncertain or fails a check.
Why Model Routing Matters
Prices between model tiers can differ several times over per token, and most tasks in a pipeline do not need the most capable model. In security triage, a large share of findings turn out to be false positives. Sending every one of them down the deepest, most expensive path spends most of the budget confirming noise.
The Risk in Routing
Routing saves money only if the cheap path is safe. A cheaper model that wrongly dismisses a real vulnerability costs far more than the tokens it saved. Good routing is designed around that asymmetry: the cheap path may close a task only with clear evidence, and anything uncertain moves up to a stronger model or a human.
Model Routing Is Ongoing Work
Model prices and abilities change every few months. A routing policy that was right last year can be wasteful or unsafe today, so routing needs its own evaluations and regular review.
How Nullify Uses Model Routing
Nullify's harness runs a mix of models and automatically routes work to cheaper ones when that is all a task needs. Cheap models investigate where the evidence settles the question, and deeper reasoning is reserved for the findings that need it. If a screening model is uncertain, runs out of budget or hits a tool error, the finding goes to full triage unchanged. See how Nullify triages findings.
Key Takeaways
- Model routing sends each task to the cheapest model that can do it reliably.
- Most pipeline tasks do not need the most capable model.
- The cheap path must only close tasks with clear evidence, and escalate when unsure.
- Routing policies need re-evaluation as model prices and abilities change.
Read More About Model Routing
Triage & Remediation
See Nullify in Action
Nullify routes work to cheaper models when they can do the job, and escalates when they cannot.