What is an LLM router?

An LLM router chooses which model should answer a request, so the calling software does not have to name one. Conifer’s router weighs what the request needs against the cost of each model in the pool you allow.

Which models can the router choose?

Only the models in your routing pool, which you set in the console under Models, Routing pool. With an empty pool it chooses from every model your key can call. A key limited to a list of named models cannot use the router.

Does the router switch to another model if the chosen one fails?

No. If the chosen model’s provider fails, the gateway serves the same model from another provider and says so on the receipt. If the router cannot make a choice, the request fails and nothing is charged.

How do I know which model answered?

Every response names the model that served it and marks the request as routed. The Conifer SDK shows the same details on each answer’s receipt.

Does the router see my request content?

Yes. To choose a model, the request content goes to the router, which may be Conifer’s router service or a decision model hosted by one of Conifer’s model providers. When you name a model, no request content reaches the router.

Router

LLM router that chooses a model for each request

Send beta-auto as the model and Conifer chooses one from a pool you approve, weighing the task against its cost. Each response names the model it chose and what the request cost.

Set up a routing pool

How the router works

The router is part of the Conifer gateway and is in beta. It acts only when a request asks for the model beta-auto. A request that names any other model is served as named and never routed.

Request: Your client asks for beta-auto on Chat Completions, Responses, legacy Completions or the Anthropic Messages interface. On Messages, the router chooses among models built for that interface.
Choice: The router reads what the request needs and picks one model from your routing pool. Only models that can hold the whole request are considered.
Serving: The gateway sends the request to a provider for that model. If the provider fails, another provider serves the same model. The router never switches to a different model on its own.
Receipt: The response names the model the router chose and what the request cost, and marks the request as routed.
Routing fee: None. A request sent to beta-auto costs only the model that answers it.

Each response names the model the router chose and what the request cost, so every automatic choice can be checked afterwards.

Your routing pool

The routing pool is the list of models the router may choose from. You set it in the console under Models, Routing pool. If the pool holds only the models your organization has approved, the router chooses only among them.

Empty pool: The router chooses from every model your key can call.
Team keys: Company keys use the team owner’s pool, so one list applies to everyone on the team.
No suitable model: If no model in the pool can serve a request, for example because none can hold a request that long, the request is refused before anything is charged.
Scoped keys: A key limited to a list of named models cannot use the router. Use a scoped key where a workload must stay on specific models.

When the router cannot choose

When the router cannot make a valid choice, the request fails and nothing is charged. It never falls back to a model of its own choosing.

No decision this time: The router could not make a valid decision. Send the request again, or name a model.
Nothing in the pool fits: No model in your pool, or in the catalog your key can call, can serve this request.
Request not accepted: The router could not read this input, or the request also listed fallback models, which the router does not accept.

When to name a model instead

Name a model when a workload has been tested and approved on that model, when you need the same model on every request, or when request content must not reach a routing service. To choose a model, the router receives the request content. That service may be Conifer’s own router or a decision model hosted by one of Conifer’s model providers, as the privacy policy sets out. A named request skips the router entirely.

Use it from any client

Asking for beta-auto is the whole integration. Any client that lets you set the model works.

curl: Send a request with beta-auto as the model. The response headers name the model that answered.
curl https://api.conifer.build/v1/chat/completions \ -H "Authorization: Bearer $CONIFER_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "beta-auto", "messages": [{"role": "user", "content": "Summarize this report"}]}'
OpenAI SDK: Point the client at Conifer and ask for beta-auto. The chosen model is in the raw response headers.
import os from openai import OpenAI client = OpenAI( base_url="https://api.conifer.build/v1", api_key=os.environ["CONIFER_API_KEY"], ) raw = client.chat.completions.with_raw_response.create( model="beta-auto", messages=messages, ) print(raw.headers["x-conifer-effective-model"]) # the model the router chose
Conifer SDK: Each answer carries its receipt, with the chosen model, in TypeScript and Python.
const answer = await conifer.chat({ model: "beta-auto", messages }); console.log(answer.receipt.effectiveModel); // the model the router chose
Codex and other agents: Add Conifer as a model provider, then ask for beta-auto. The Codex page has the configuration.

The router reference has full examples, and Routing covers named requests, providers and failover.

What is an LLM router?: How a router picks a model for each request, and what it should tell you afterwards.
Conifer Gateway: One API key and endpoint for every model in the catalog, with the model and cost stated on every response.
What is an LLM gateway?: How a gateway sits between your software and the companies that serve models, and what to check when you choose one.
All comparisons: Conifer compared with eleven other LLM gateways and routers, from their own documentation.

Questions

What is an LLM router?: An LLM router chooses which model should answer a request, so the calling software does not have to name one. Conifer’s router weighs what the request needs against the cost of each model in the pool you allow.
Which models can the router choose?: Only the models in your routing pool, which you set in the console under Models, Routing pool. With an empty pool it chooses from every model your key can call. A key limited to a list of named models cannot use the router.
Does the router switch to another model if the chosen one fails?: No. If the chosen model’s provider fails, the gateway serves the same model from another provider and says so on the receipt. If the router cannot make a choice, the request fails and nothing is charged.
How do I know which model answered?: Every response names the model that served it and marks the request as routed. The Conifer SDK shows the same details on each answer’s receipt.
Does the router see my request content?: Yes. To choose a model, the request content goes to the router, which may be Conifer’s router service or a decision model hosted by one of Conifer’s model providers. When you name a model, no request content reaches the router.