skip to content
Serve an endpoint

Local

Serve an endpoint

conifer serve is the local endpoint your tools already point at. On macOS and Windows it can serve a model on your machine through the same port.

terminal
conifer serve                        # 127.0.0.1:8080 in front of the gateway
conifer serve --model <local-model>  # the same port, with a local model as the default

conifer serve listens on 127.0.0.1:8080 and speaks the same OpenAI and Anthropic wires as api.conifer.build. A tool points at it once. A request that names a gateway model goes to the gateway. A request that names a model on this machine is answered here.

terminal
curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"<local-model>","max_tokens":64,"messages":[{"role":"user","content":"ping"}]}'

Lifecycle

--ttl <seconds> unloads a local model after it has sat idle that long. The default keeps it loaded. --max-loaded caps how many stay in memory at once. --detach runs the server in the background, and conifer host stop ends it.

Serve on the LAN

terminal
conifer serve peers mint <name>   # one access token per machine that will connect
conifer serve --bind lan

On loopback there is no auth. To serve to other machines, mint a token for each one first; each can be revoked on its own. This is how a Mac or a Windows machine serves a model to a Linux box on the same network.

What stays on the machine

A request to a local model never leaves the machine. The prompt is run here and the answer streams from here. What can go out is a gateway request you asked for, a model download you started, a daily update check, and usage telemetry, which carries counts and identifiers, never your prompts or answers. conifer privacy opt-out turns telemetry off and conifer privacy offline on stops all background traffic.