Engineering

How ORION decides which model to use

· 5 min read

A single request rarely needs a single model. Reading a screen, drafting a plan, extracting a value from a table, and writing a reply have different requirements. Treating them as one prompt to one provider wastes latency on the easy steps and under-serves the hard ones.

Four inputs to every routing decision

Capability: can the model reliably do this specific step, judged on our own evaluations rather than a leaderboard.

Latency: a step that blocks the interface has a stricter budget than one that runs in the background.

Privacy: if a step touches content you marked as sensitive, it stays on device or does not run.

Cost: when two options are equivalent for a step, the cheaper one wins.

Legibility is a requirement

Routing that you cannot inspect is indistinguishable from routing that is wrong. ORION records the model chosen for each step and the reason, and that record is part of the product rather than a debug view.

This layer is being built now. It is not finished, and the on-device half is still an open research question for us.