Models
Reading a model
What the numbers on a catalog entry mean, and how to pick a model for the job.
Every model behind the gateway is listed on the models page and at GET /v1/catalog with its id, its lab, its context window, its price per million tokens in and out, and what it accepts. This page explains how to read that entry. The catalog itself is on The catalog.
How to choose
Start from the task, not the leaderboard. Then let the gateway handle the provider.
| If you are doing | Reach for |
|---|---|
| Writing or reasoning about code | a strong instruct model, or a reasoning model for a hard multi-file change |
| Math, proofs, hard multi-step problems | a reasoning model that thinks before it answers |
| Drafting, editing and conversation | a well-rounded instruct model |
| Summaries, extraction, classification at volume | a small, fast instruct model. The price gap is large and the quality gap usually is not |
| Reading images or documents | a model whose entry lists vision |
| Inline autocomplete | a small code model, or an open model on your machine |
If you would rather not choose, a route lets the gateway pick per request. See Routing.
The four numbers
- context window
- How many tokens the model can hold at once, prompt and answer together. A request that exceeds it is refused before anything is billed.
- price
- Per million tokens, in and out, as published in the catalog. The receipt on every response tells you what a call actually cost.
- intelligence
- A published benchmark score. The ledger uses the Artificial Analysis Intelligence Index, a composite of several evals. Treat it as a guide, not a guarantee.
- speed
- Tokens per second while the answer streams. It varies by provider, which is one of the things the gateway takes into account when it picks one.
Instruct and reasoning
An instruct model follows instructions, holds a conversation and uses tools. A model listed with no qualifier is instruct, and it is the right default. A reasoning model works through the problem before it answers. That pays off on math, logic and difficult code, and is wasted on a one-line question. Many entries offer both, and on the gateway you set the reasoning effort per request.
| Tuning | Reach for it when | The name shows |
|---|---|---|
| Instruct | almost everything: chat, code, tasks, tools | no suffix · -Instruct · -it |
| Reasoning | math, proofs, logic, hard multi-step problems | -thinking · R1 · a reasoning effort setting |
| Base | raw completion only. Not for chat | -base |
Open models on your machine
The CLI can also run open models on Apple Silicon and Windows. Two more numbers matter there, and neither applies to a gateway model.
Size
An open model is measured in billions of parameters, and at the usual 4-bit precision it needs a little over half a gigabyte of memory per billion. Leave room for your other apps. A fast small model beats a large one that crawls.
| You have | Start with |
|---|---|
| 8 to 16 GB | a 3 to 4B model |
| 16 to 32 GB | an 8 to 12B model |
| 32 GB and up | a 24 to 32B model |
conifer setup picks one for you from what fits.
Quantization
Models are trained at 16 bits per weight and shipped for local use at fewer. GGUF is the file format Conifer runs. A name like Q4_K_M is the recipe: 4 bits per weight, the default and near-full quality. Q6_K and Q8_0 are larger and closer to the original. Spend memory on a stronger model first and on precision second.
What runs
Anything in conifer models search runs. A GGUF you import yourself runs when its architecture is one the engine supports. The CLI tells you at import time if it is not.
conifer models search qwen
conifer models pull <build-name>The rest is on Get models.