Sending each request to the cheapest model that can handle it.
Routing classifies requests by difficulty and sends easy ones to a low-cost model while reserving frontier models for complex work. A fallback path handles failures and low-confidence answers.
Most production traffic is not uniformly hard. Routing is often the largest structural cost saving after prompt caching.