> ## Documentation Index
> Fetch the complete documentation index at: https://developers.telnyx.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Routing and Fallbacks

> Retries, timeouts, and fallbacks to other models, set on each request.

> [!WARNING]
>
> **Beta.** An instance sets no routing of its own. Each request sets its own fallbacks, retries, and timeout, as on this page.

A request can retry when it fails, limit how long each attempt takes, and fall back to other models.

## Retries

An instance retries a failed request only when the request sets `num_retries`. The retries run before the request fails or falls back.

A model that keeps failing is taken out of rotation for 5 seconds, and requests for it return `429` meanwhile.

## Fallbacks

When a request fails and its retries are spent, the gateway tries the request's fallbacks in order. When every fallback fails, the response is the first model's error.

## Per-Request Settings

A request can set its own routing in the request body:

| Field | What It Does |
| - | - |
| `fallbacks` | The models to try, in order, if this request fails. An entry can be a model name, or an object with `model` and parameters that replace the request's own for that model. |
| `disable_fallbacks` | `true` skips the request's fallbacks. |
| `num_retries` | How many times to retry the request. |
| `timeout` | How long each attempt may take, in seconds. Without it, an attempt that is not streamed fails after 30 seconds. |
| `stream_timeout` | The same, for a streamed request. |

A fallback model must be on the key's [allow list](https://developers.telnyx.com/docs/agent-gateway/authorization/allow-lists.md), or the gateway refuses the request.

## Fallback Example

This request asks `GLM-5.3-Flash` for more output tokens than it allows. Its fallback, `Qwen3.8-27B-nothink`, replaces `max_tokens` with `8`:

```bash Host theme={null}
curl "$GATEWAY_URL/v1/chat/completions" \
  -H "Authorization: Bearer $AGW_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "GLM-5.3-Flash",
    "max_tokens": 100000000,
    "messages": [{ "role": "user", "content": "Say hi" }],
    "fallbacks": [{ "model": "Qwen3.8-27B-nothink", "max_tokens": 8 }]
  }'
```

`GLM-5.3-Flash` refused the request with `422`, and the fallback answered. After a fallback, `model` holds the Telnyx inference model that answered:

```json Output theme={null}
{
  "id": "chatcmpl-10acb933-e5c5-42aa-aea4-82549133320a",
  "model": "Qwen/Qwen3.8-27B",
  "object": "chat.completion",
  "choices": [
    {
      "finish_reason": "length",
      "message": { "content": "Hi! How can I help you today", "role": "assistant", ... },
      ...
    }
  ],
  "usage": { "completion_tokens": 8, "prompt_tokens": 14, "total_tokens": 22, ... },
  ...
}
```

Without `fallbacks`, or with `"disable_fallbacks": true`, the same request returns the `422`.