Does an LLM router save money?

It can, when many requests are simple enough for a cheaper model. The saving depends on your traffic and on how often the router picks a cheaper model without hurting the answer, so compare its records against the models you would otherwise name.

Is an LLM router the same as load balancing?

No. Load balancing spreads requests for one model across providers or deployments. A router chooses between different models.

Can I limit which models a router uses?

That depends on the router, so check before you rely on it. Conifer’s router picks only from the routing pool you set in the console. With an empty pool it can pick any model your key can call.

How do I know which model the router chose?

Check whether the router reports it. Conifer names the model that answered on every response and marks the request as routed.

Learn

What is an LLM router?

An LLM router chooses which AI model should answer each request, so the calling software does not have to name one. This page explains how routers decide, how routing differs from fallbacks, and what to check before you use one.

See the Conifer router

What an LLM router does

AI models differ in price, speed and skill. A short factual question rarely needs the most capable model, and a hard coding task may fail on a small one. A router reads each request and picks a model from a set you allow, aiming for a good enough answer at a lower cost, or the best answer within a budget.

Reading the request: The router looks at what the request needs: the prompt, its length, whether it uses tools or images, and sometimes the conversation so far.
Choosing: Models that cannot hold the request, or lack a feature it needs, are left out. The router picks one of the rest.
Serving: The request goes to the chosen model, usually through a gateway that holds the keys and tries another provider if one fails.
Reporting: A router should say which model it picked, so each choice can be checked and each cost explained.

A routed request should still say which model answered and what it cost, so each automatic choice can be checked afterwards.

How routers decide

Routers use one of a few methods, or a mix of them.

Rules: Fixed rules send requests to models by length, by the calling application or by a tag. Rules are predictable and easy to audit, but they do not adapt to what a request contains.
Classifiers: A small model labels each request, for example by task type or difficulty, and each label maps to a model.
Learned routers: A model trained on past requests and their results predicts which candidate will answer well for the least cost. The open-source RouteLLM framework, for example, ships trained routers with a cost threshold that sets the trade-off between cost and quality.
Cascades: The request goes to a cheaper model first and moves to a stronger one if the first answer fails a check. This saves money on easy requests and adds time on hard ones.

Routing is not the same as fallback

A router chooses a model before the request is sent. A fallback moves a request to another model after something fails. Load balancing is different again: it spreads requests for one model across providers or deployments. Some systems combine all three, so if a workload has been tested on specific models, check which behaviour you will get.

What to check before you use one

Allowed models: Whether you can limit the router to the models your organization has approved.
Disclosure: Whether each response names the model that answered.
Failures: What happens when the router cannot choose, or the chosen model fails: an error, a default model, or another model picked for you.
Request content: Whether the content of a request leaves your environment for the routing decision, and who processes it.
Cost: Whether the router charges for each decision, and whether its records let you measure the saving against the models you would otherwise name.
When to name a model: Name a model when a workload has been tested and approved on it, or when you need the same model on every request.

How the Conifer router works

The Conifer router is part of the Conifer gateway and is in beta. Send beta-auto as the model, and it picks one from your routing pool, weighing what the request needs against the cost of each model. The router page has the details.

Routing pool: You set the models it may choose from in the console. With an empty pool it chooses from every model your key can call, and team keys use the team owner’s pool.
Receipt: Every response names the model the router chose and what the request cost, and marks the request as routed.
curl https://api.conifer.build/v1/chat/completions \ -H "Authorization: Bearer $CONIFER_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "beta-auto", "messages": [{"role": "user", "content": "Summarize this report"}]}'
Failures: If the router cannot make a valid choice, the request fails and nothing is charged. It never falls back to a model of its own choosing. If the chosen model’s provider fails, another provider serves the same model.
Request content: To choose a model, the router receives the request content. It may go to Conifer’s own router or to a decision model hosted by one of Conifer’s model providers, as the privacy policy sets out. A request that names a model skips the router.

The router reference has examples for curl, the Conifer SDK and the OpenAI SDK, and Routing covers named requests and failover.

Conifer Router: Send one model name and Conifer chooses a model for each request from a pool you approve.
What is an LLM gateway?: How a gateway sits between your software and the companies that serve models, and what to check when you choose one.
Conifer vs OpenRouter: The two gateways side by side, from each company's own documentation.
All comparisons: Conifer compared with eleven other LLM gateways and routers, from their own documentation.

Questions

What is an LLM router?: An LLM router chooses which AI model should answer a request. It reads what the request needs and picks a model from a set you allow, usually to lower cost while keeping answers good enough.
Does an LLM router save money?: It can, when many requests are simple enough for a cheaper model. The saving depends on your traffic and on how often the router picks a cheaper model without hurting the answer, so compare its records against the models you would otherwise name.
Is an LLM router the same as load balancing?: No. Load balancing spreads requests for one model across providers or deployments. A router chooses between different models.
Can I limit which models a router uses?: That depends on the router, so check before you rely on it. Conifer’s router picks only from the routing pool you set in the console. With an empty pool it can pick any model your key can call.
How do I know which model the router chose?: Check whether the router reports it. Conifer names the model that answered on every response and marks the request as routed.