Operate
Failover and the first byte
MiniRouter can retry another provider before output begins. Once streaming starts, the response stays with that provider.
Automatic provider retries
If the preferred host times out or errors before your response starts, the request moves to the next healthy host. You see a slower first token and nothing else: same model, same ID. To allow a different model, choose Auto or send an explicit fallback list. The x-minirouter-provider header on every response names who actually served it.
Once output begins
MiniRouter can retry another provider before the first content byte reaches you. Once the first byte is sent, provider switching is impossible. Splicing a second provider’s tokens into a half-delivered completion corrupts output, tool-call framing, and your client’s parser state.
| Event | Before first byte | After first byte |
|---|---|---|
| TTFT timeout | Ends with request_timeout. Check Activity before retrying | Cannot occur. The first byte arrived. |
| Upstream 500 | Retry. You pay only for the successful attempt | Terminal error. What was produced is settled. |
| Fallback chain | Live. The full route list is available | Closed. The route is committed. |
You pay only for the attempt that succeeded. After the first byte, an upstream failure ends the stream with a terminal error. Under 50 output tokens, the charge is written off entirely.
Time to first token
Time to first token measures how long you wait for output to begin. Compare providers on each model page.
Errors on this path
request_timeoutThe first-token or complete-response deadline expired. Check Activity before retrying; the original request may still incur a charge.upstream_errorThe upstream failed after streaming began, past the point of retry.model_unavailableNo provider could serve this model. Try an alternative named in the error.