Streaming

Server-sent events, unbuffered end to end.

A streaming request
curl -N https://api.minirouter.sh/v1/chat/completions \
  -H "Authorization: Bearer $MINIROUTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta/llama-3.3-70b",
    "messages": [{"role":"user","content":"ping"}],
    "stream": true
  }'

Set "stream": true and read SSE chunks until data: [DONE]. The reverse proxy flushes every chunk immediately.

The cost trailer

The final chunk before [DONE] carries the usage object, including usage.cost — the exact USD charge:

Final chunks of a stream
data: {"id":"chatcmpl-...","choices":[{"delta":{},"finish_reason":"stop","index":0}],
       "usage":{"prompt_tokens":12480,"completion_tokens":902,
                "total_tokens":13382,"cost":0.0412}}

data: [DONE]

Non-streaming responses carry the same field, plus the x-minirouter-cost-usd, -provider, -ttft-ms and -balance-usd headers.

Aborting a stream

When your client closes the socket we abort the upstream immediately. You are charged for the tokens generated before the abort — the provider billed us regardless — recorded usage_source=estimated_on_abort. A stream the upstream kills before 50 output tokens is written off entirely.

Mid-stream exhaustion

If your balance reaches $0 mid-stream, the stream ends with an OpenAI-shaped error chunk, then [DONE], then a clean close — never a silent socket close:

Terminal exhaustion chunk
data: {"error":{"message":"Insufficient credits: this response stopped because
       your balance reached $0. You were charged $0.0031 for the tokens
       delivered. Top up at https://minirouter.sh/dashboard/billing",
       "type":"invalid_request_error","code":"insufficient_credits"}}

data: [DONE]

Details: insufficient_credits. x-minirouter-balance-usd is on every response so your tooling can warn first.

Retries and the first byte

Before the first content byte we retry on another upstream automatically; after it, we cannot. See failover.