# Telnyx AI: AI Gateway — Full Documentation
> Complete page content for AI Gateway (AI section) of the Telnyx developer docs (https://developers.telnyx.com).
> This file: https://developers.telnyx.com/docs/development/llms/ai-ai-gateway-llms-full-txt · Root index: https://developers.telnyx.com/llms.txt

## AI Gateway

### Overview

> Source: https://developers.telnyx.com/docs/inference/ai-gateway.md

AI Gateway issues **scoped inference credentials** for applications. Instead of sharing a Telnyx account API key with every service, create a token group, issue a token key inside it and hand that key to the application. The gateway enforces the model allowlist, budget and rate limits attached to the key, and records every request in a usage ledger attributed to the key, its user, its group and the end user it served.

Applications call the gateway with the official OpenAI SDKs by changing the base URL and the API key. No request rewriting is required. Anthropic models used with your own Anthropic key can also be called with the official Anthropic SDKs.

## Capabilities

| Capability | Behavior |
| --- | --- |
| Token groups | Define model access, group budgets and rate limits. |
| Token users | Associate an application actor with one or more groups; aggregate its limits across all of its keys. |
| Token keys | Issue scoped inference credentials for a user or a service; narrow model access, set limits and expiry, revoke access. |
| End users | Apply account-scoped budget caps and blocks to a caller-asserted application user identifier. |
| Provider keys (BYOK) | Store an OpenAI or Anthropic key once and attach it to groups; your provider bills requests on bring-your-own-key models. |
| Inference | OpenAI Chat Completions, Anthropic Messages (Anthropic BYOK models), [model discovery](/docs/inference/ai-gateway/inference-api#models) and streaming. |
| Usage | Durable request accounting with reservations, corrections and dimensional summaries. |

## Two planes, two credentials

The gateway exposes a **management plane** for provisioning and reporting, and an **inference plane** that applications call. They use different hostnames and different credentials.

| Plane | Base URL | Credential |
| --- | --- | --- |
| Management | `https://api.telnyx.com/v2/llm_token_gateway` | Telnyx account API key: `Authorization: Bearer $TELNYX_API_KEY` |
| OpenAI-compatible inference | `https://llm.telnyx.com/v1` | AI Gateway token key: `Authorization: Bearer $AI_GATEWAY_TOKEN_KEY` |
| Anthropic-compatible inference (Anthropic BYOK models) | `https://llm.telnyx.com` as the SDK base URL | Same token key via `x-api-key`; the SDK appends `/v1/messages` |

Token keys start with `ltg_sk_`. The inference plane rejects Telnyx account API keys and provider secrets; the management plane rejects token keys. Keep every credential in a trusted backend or secret store, never in browser code, source control or logs.

## How it works

1. **Create a token group** with an explicit `allowed_models` list and optional budget and rate limits. To use your own OpenAI or Anthropic account, attach a [provider key](/docs/inference/ai-gateway/byok) to the group.
2. **Issue a token key** in that group, optionally bound to a token user. The secret is returned once, on the create response.
3. **Call models** from the application with an OpenAI SDK (or, for Anthropic BYOK models, an Anthropic SDK) pointed at the inference base URL and authenticated with the token key.
4. **Inspect usage** with the spend events and spend summary endpoints, filtered by group, user, key or end user.
5. **Revoke** the key when the application no longer needs it. New admissions stop immediately; spend history is retained.

Telnyx owns authorization, admission, revocation and the authoritative usage ledger. Applications never hold provider credentials or configure model providers directly.

Usage of Telnyx-hosted models is billed to your Telnyx account at standard Telnyx AI Inference pricing for each model. Requests on bring-your-own-key models are billed by your provider on your provider account. Budgets and reported `cost` values use a flat reference rate for enforcement and attribution; see [Budgets](/docs/inference/ai-gateway/controls#budgets).

## Next steps

    Create a group, issue a token key, make a request and revoke the key.
    Telnyx-hosted and BYOK models, OpenAI and Anthropic SDK configuration, streaming, request limits and supported fields.
    Groups, users, keys, end users and provider keys, with idempotency and ETag rules.
    How each control is enforced and what happens at the limit.
    Use your own OpenAI or Anthropic key; your provider bills those requests.
    Spend events, dimensional summaries and snapshot pagination.
    Status codes, structured error codes and how to handle them.

---

### Quickstart

> Source: https://developers.telnyx.com/docs/inference/ai-gateway/quickstart.md

This quickstart provisions a group and a service token key through the management API, makes an inference request with that key, reads the resulting usage and revokes the key. Every step is a copy-paste request.

## Prerequisites

- A Telnyx API key from the [portal](https://portal.telnyx.com/#/api-keys).
- A model name from the [available models](/docs/inference/ai-gateway/inference-api#models). This guide uses `Kimi-K3`. The inference `GET /v1/models` endpoint is scoped to a token key, so it cannot be used to discover models before a key exists.
- `curl`, `jq` and `uuidgen` (or another way to generate a UUID for the `Idempotency-Key` header).

Export the credential, the model name and the two base URLs so the examples work as-is:

```bash
export TELNYX_API_KEY="KEY..."
export AI_GATEWAY_MODEL="Kimi-K3"
export AI_GATEWAY_MANAGEMENT_BASE_URL="https://api.telnyx.com/v2/llm_token_gateway"
export AI_GATEWAY_INFERENCE_BASE_URL="https://llm.telnyx.com/v1"
```

  This walkthrough creates persistent account resources and can incur model charges.

A token group defines which models its keys may call and the limits they share. `name` and `allowed_models` are required. An empty `allowed_models` list permits no inference.

This example sets a USD 10 budget per anchored one-day period and 60 requests per minute.

```bash
curl -X POST "$AI_GATEWAY_MANAGEMENT_BASE_URL/token_groups" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d "$(jq -n --arg model "$AI_GATEWAY_MODEL" '{
    name: "support-assistant",
    allowed_models: [$model],
    max_budget: 10,
    budget_duration: "1d",
    rpm_limit: 60
  }')"
```

```json
{
  "data": {
    "record_type": "token_group",
    "id": "5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33",
    "name": "support-assistant",
    "allowed_models": ["Kimi-K3"],
    "max_budget": 10,
    "budget_duration": "1d",
    "rpm_limit": 60,
    "tpm_limit": null,
    "provider_key_ids": [],
    "blocked": false,
    "spend": 0,
    "reserved_spend": 0,
    "budget_started_at": "2026-09-22T10:00:00Z",
    "resets_at": "2026-09-23T10:00:00Z",
    "version": 1,
    "created_at": "2026-09-22T10:00:00Z",
    "updated_at": "2026-09-22T10:00:00Z"
  }
}
```

Keep `data.id`; the next step needs it and the group is what you retire at the end.

```bash
export GROUP_ID="5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33"
```

Every mutation requires an `Idempotency-Key`. If a request times out, retry it with the **same** key and body; a new key starts a different operation and can create a second resource.

A token key is the credential the application uses. `token_user_id: null` creates a **service key** owned by the group alone. Null `allowed_models` inherits the group's list. Key limits that are omitted are set to their maximums: USD 1,000 lifetime budget, 6,000 requests and 10,000,000 tokens per minute. See [Token key limits](/docs/inference/ai-gateway/controls#token-key-limits).

```bash
curl -X POST "$AI_GATEWAY_MANAGEMENT_BASE_URL/token_keys" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d "$(jq -n --arg group "$GROUP_ID" '{
    name: "support-backend",
    token_group_id: $group,
    token_user_id: null
  }')"
```

```json
{
  "data": {
    "record_type": "token_key",
    "id": "9b7e4a10-3c2d-4f5e-8a6b-1d2c3e4f5a60",
    "name": "support-backend",
    "token_group_id": "5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33",
    "token_user_id": null,
    "allowed_models": null,
    "max_budget": 1000,
    "budget_duration": null,
    "rpm_limit": 6000,
    "tpm_limit": 10000000,
    "expires_at": null,
    "blocked": false,
    "required_end_user_id": false,
    "spend": 0,
    "reserved_spend": 0,
    "budget_started_at": null,
    "resets_at": null,
    "version": 1,
    "created_at": "2026-09-22T10:01:00Z",
    "updated_at": "2026-09-22T10:01:00Z",
    "token": "ltg_sk_..."
  }
}
```

  `data.token` is returned **only on the original create response**. GET, list and idempotent replays return metadata without the secret. If the response is lost, revoke the key and create a new one with a new idempotency key.

Store the token in your secret store now, then export it and the key ID for the remaining steps:

```bash
export TOKEN_KEY_ID="9b7e4a10-3c2d-4f5e-8a6b-1d2c3e4f5a60"
export AI_GATEWAY_TOKEN_KEY="ltg_sk_..."
```

To issue a key for a specific application user instead, first `POST /token_users` with `name` and `token_group_ids: [GROUP_ID]`, then pass the returned ID as `token_user_id`. Key ownership cannot be changed later; issue a replacement key instead.

The inference plane authenticates with the token key, not the account API key. `GET /v1/models` returns only the models this key may use.

```bash
curl "$AI_GATEWAY_INFERENCE_BASE_URL/models" \
  -H "Authorization: Bearer $AI_GATEWAY_TOKEN_KEY"
```

Send a Chat Completions request with one of those models:

```bash curl
curl -X POST "$AI_GATEWAY_INFERENCE_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_TOKEN_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg model "$AI_GATEWAY_MODEL" '{
    model: $model,
    messages: [{role: "user", content: "Hello"}],
    max_tokens: 32
  }')"
```

```python Python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AI_GATEWAY_TOKEN_KEY"],
    base_url=os.environ["AI_GATEWAY_INFERENCE_BASE_URL"],
    max_retries=0,
)

completion = client.chat.completions.create(
    model=os.environ["AI_GATEWAY_MODEL"],
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=32,
)
print(completion.choices[0].message.content)
```

```javascript JavaScript
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_TOKEN_KEY,
  baseURL: process.env.AI_GATEWAY_INFERENCE_BASE_URL,
  maxRetries: 0,
});

const completion = await client.chat.completions.create({
  model: process.env.AI_GATEWAY_MODEL,
  messages: [{ role: "user", content: "Hello" }],
  max_tokens: 32,
});
console.log(completion.choices[0].message.content);
```

Inference requests are **not idempotent**. The examples disable SDK retries so that a timeout cannot silently trigger a second billed request. Set `max_tokens` on every request: a request without it reserves the model's full output allowance against budgets and `tpm_limit`. See [Inference API](/docs/inference/ai-gateway/inference-api) for streaming and request size limits.

Spend events are queried over a half-open UTC date range `[start_date, end_date)` of at most 31 days, using ISO dates rather than timestamps. Filter by the key you just used:

```bash
curl --globoff -G "$AI_GATEWAY_MANAGEMENT_BASE_URL/spend/events" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  --data-urlencode "start_date=2026-09-22" \
  --data-urlencode "end_date=2026-09-23" \
  --data-urlencode "token_key_id=$TOKEN_KEY_ID" \
  --data-urlencode "page[size]=100"
```

```json
{
  "data": [
    {
      "id": "c0a1b2c3-d4e5-4f60-8a71-92b3c4d5e6f7",
      "request_id": "0b1c2d3e-4f50-4617-8a29-3b4c5d6e7f80",
      "created_at": "2026-09-22T10:02:14Z",
      "token_group_id": "5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33",
      "token_user_id": null,
      "token_key_id": "9b7e4a10-3c2d-4f5e-8a6b-1d2c3e4f5a60",
      "end_user_id": null,
      "model": "Kimi-K3",
      "input_tokens": 9,
      "output_tokens": 12,
      "cost": 0.000225,
      "status": "succeeded",
      "usage_status": "known",
      "configuration_version": 1,
      "rate_version": "telnyx-reference-v1",
      "reservation_micro_usd": 0
    }
  ],
  "meta": {
    "page_number": 1,
    "page_size": 100,
    "has_more": false,
    "snapshot": "..."
  }
}
```

When `meta.has_more` is true, request the next `page[number]` with the same filters and `page[snapshot]=<meta.snapshot>`. See [Usage reporting](/docs/inference/ai-gateway/usage) for summaries grouped by group, user, key or end user.

`DELETE` requires the resource's current `ETag` in `If-Match`. Read the key first; the metadata GET never reveals the token.

```bash
KEY_ETAG=$(curl -sS -I "$AI_GATEWAY_MANAGEMENT_BASE_URL/token_keys/$TOKEN_KEY_ID" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  | awk 'tolower($1) == "etag:" { sub(/\r$/, "", $2); print $2 }')

curl -X DELETE "$AI_GATEWAY_MANAGEMENT_BASE_URL/token_keys/$TOKEN_KEY_ID" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "If-Match: $KEY_ETAG"
```

A `204` means the key is revoked: new requests with that token are rejected, while requests already admitted may finish. Spend history is retained. On `412`, the resource changed since the read; fetch it again rather than forcing the write.

The group stays available for new keys. To retire it as well, `DELETE /token_groups/{id}` with a fresh ETag and idempotency key. Deleting a group cascades to its keys.

## Next: use your own provider key

To run OpenAI or Anthropic models on your own provider account, store the provider key once, attach it to a group and allow a [bring-your-own-key model](/docs/inference/ai-gateway/inference-api#bring-your-own-key-models). Applications keep using the same token key. See [Bring your own key](/docs/inference/ai-gateway/byok).

## Next steps

    Available models, streaming, request limits and the supported request fields.
    Users, end users, PATCH semantics and pagination.
    What each limit does and how it is enforced.
    Status codes and structured error codes on both planes.
    Attach your own OpenAI or Anthropic key to a group.

---

## API

### Inference API

> Source: https://developers.telnyx.com/docs/inference/ai-gateway/inference-api.md

The inference plane serves three endpoints. All of them authenticate with an AI Gateway token key (`ltg_sk_...`) issued through the [management API](/docs/inference/ai-gateway/management-api). Telnyx account API keys, provider secrets and any other credential are rejected on this plane.

| Endpoint | Purpose |
| --- | --- |
| `GET https://llm.telnyx.com/v1/models` | Models available to the supplied token key. |
| `POST https://llm.telnyx.com/v1/chat/completions` | OpenAI-compatible Chat Completions, including streaming. Works with every model. |
| `POST https://llm.telnyx.com/v1/messages` | Anthropic-compatible Messages, including streaming. Only for [Anthropic BYOK models](#anthropic-sdk). |

Compatibility is bounded by the AI Gateway contract, not by every option the OpenAI and Anthropic SDKs expose; see [Supported request fields](#supported-request-fields) and [Supported Messages fields](#supported-messages-fields).

## Models

These Telnyx-hosted models are billed to your Telnyx account. Use these model names in the `model` field:

- `Kimi-K3`
- `Kimi-K2.6`
- `Kimi-K2.5`
- `GLM-5.3`
- `GLM-5.3-Flash`
- `GLM-5.2`
- `GLM-5.1-FP8`
- `MiniMax-M3-MXFP8`
- `MiniMax-M2.7`
- `Qwen3.8-27B`
- `Qwen3-235B-A22B`
- `DeepSeek-V4.1-Flash`
- `DeepSeek-V4-Flash-0731`
- `Llama-3.3-70B-Instruct`
- `Meta-Llama-3.1-70B-Instruct`
- `Meta-Llama-3.1-8B-Instruct`
- `gemma-2b-it`

A group lists the model names its keys may call in `allowed_models`; a key can narrow that list further. Per-request output and prompt limits are listed under [Request size limits](#request-size-limits).

### Bring-your-own-key models

These models run on your own provider account with a [provider key](/docs/inference/ai-gateway/byok) and are billed by that provider.

| Provider | Models |
| --- | --- |
| Anthropic | `claude-fable-5-1`, `claude-opus-5-5`, `claude-sonnet-5` |
| OpenAI | `gpt-6-astra`, `gpt-6-sol`, `gpt-6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4o`, `gpt-4o-mini`, `o3` |

Add them to `allowed_models` like any other model. A BYOK model works only when the calling key's group has an attached provider key for that model's provider. Without one, requests fail with `503` and code `enforcement_unavailable`; they never fall back to a Telnyx-hosted model.

Every BYOK model works on `/v1/chat/completions`. Anthropic BYOK models also work on `/v1/messages`.

### Model discovery

`GET /v1/models` returns the models the supplied key may use: the intersection of the key's `allowed_models` and its group's list. It is not a global catalog, and it does not charge against any budget.

```bash
curl https://llm.telnyx.com/v1/models \
  -H "Authorization: Bearer $AI_GATEWAY_TOKEN_KEY"
```

```json
{
  "object": "list",
  "data": [
    { "id": "Kimi-K3", "object": "model", "created": 1758535200, "owned_by": "telnyx" }
  ]
}
```

## OpenAI SDK

Point the OpenAI SDK at the inference base URL and use the token key as the API key.

```python Python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AI_GATEWAY_TOKEN_KEY"],
    base_url="https://llm.telnyx.com/v1",
    max_retries=0,
)

completion = client.chat.completions.create(
    model=os.environ["AI_GATEWAY_MODEL"],
    messages=[{"role": "user", "content": "Tell me about Telnyx"}],
    max_tokens=256,
    user="application-bound-user-id",
)
print(completion.choices[0].message.content)
```

```javascript JavaScript
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_TOKEN_KEY,
  baseURL: "https://llm.telnyx.com/v1",
  maxRetries: 0,
});

const completion = await client.chat.completions.create({
  model: process.env.AI_GATEWAY_MODEL,
  messages: [{ role: "user", content: "Tell me about Telnyx" }],
  max_tokens: 256,
  user: "application-bound-user-id",
});
console.log(completion.choices[0].message.content);
```

```bash curl
curl -X POST https://llm.telnyx.com/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_TOKEN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Kimi-K3",
    "messages": [{"role": "user", "content": "Tell me about Telnyx"}],
    "max_tokens": 256,
    "user": "application-bound-user-id"
  }'
```

The optional `user` field attributes the request to an application end user for budgeting and reporting. See [End-user identity](/docs/inference/ai-gateway/controls#end-user-identity).

### Streaming

Set `stream: true` and consume the stream to completion. A stream that starts with HTTP 200 can still fail later; handle SDK exceptions and close the stream.

```python Python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AI_GATEWAY_TOKEN_KEY"],
    base_url="https://llm.telnyx.com/v1",
    max_retries=0,
)

with client.chat.completions.create(
    model=os.environ["AI_GATEWAY_MODEL"],
    messages=[{"role": "user", "content": "Tell me about Telnyx"}],
    max_tokens=256,
    stream=True,
) as stream:
    for chunk in stream:
        if chunk.choices:
            print(chunk.choices[0].delta.content or "", end="", flush=True)
```

```javascript JavaScript
const stream = await client.chat.completions.create({
  model: process.env.AI_GATEWAY_MODEL,
  messages: [{ role: "user", content: "Tell me about Telnyx" }],
  max_tokens: 256,
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```

## Anthropic SDK

  `POST /v1/messages` is available **only for Anthropic models used with your own Anthropic key** ([Anthropic BYOK models](#bring-your-own-key-models)). A request to `/v1/messages` with a Telnyx-hosted model or an OpenAI BYOK model returns `400` with code `invalid_request`; call those models on `/v1/chat/completions`. Anthropic BYOK models work on both endpoints.

The Anthropic SDK appends `/v1/messages` to its base URL, so set the base URL to the gateway root **without** the `/v1` suffix. The SDK sends the token key in `x-api-key`, which the gateway accepts. `Authorization: Bearer` is also accepted; if both headers are present they must carry the same token.

```python Python
import os
from anthropic import Anthropic

client = Anthropic(
    api_key=os.environ["AI_GATEWAY_TOKEN_KEY"],
    base_url="https://llm.telnyx.com",
    max_retries=0,
)

message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=256,
    messages=[{"role": "user", "content": "Tell me about Telnyx"}],
    metadata={"user_id": "application-bound-user-id"},
)
print(message.content[0].text)
```

```javascript JavaScript
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_TOKEN_KEY,
  baseURL: "https://llm.telnyx.com",
  maxRetries: 0,
});

const message = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 256,
  messages: [{ role: "user", content: "Tell me about Telnyx" }],
  metadata: { user_id: "application-bound-user-id" },
});
console.log(message.content[0].text);
```

```bash curl
curl -X POST https://llm.telnyx.com/v1/messages \
  -H "x-api-key: $AI_GATEWAY_TOKEN_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Tell me about Telnyx"}],
    "metadata": {"user_id": "application-bound-user-id"}
  }'
```

Raw HTTP requests must send `anthropic-version: 2023-06-01`; the SDKs add it automatically. `metadata.user_id` and the OpenAI `user` field identify the same end-user namespace.

The endpoint streams standard Anthropic server-sent events (`message_start`, `content_block_start`, `content_block_delta`, `content_block_stop`, `message_delta`, `message_stop`). An error after the stream has started arrives as an `event: error` frame.

```python Python
with client.messages.stream(
    model="claude-sonnet-5",
    max_tokens=256,
    messages=[{"role": "user", "content": "Tell me about Telnyx"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
```

```javascript JavaScript
const stream = client.messages.stream({
  model: "claude-sonnet-5",
  max_tokens: 256,
  messages: [{ role: "user", content: "Tell me about Telnyx" }],
});

for await (const event of stream) {
  if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
    process.stdout.write(event.delta.text);
  }
}
```

### Supported Messages fields

| Field | Notes |
| --- | --- |
| `model` | Required. An Anthropic BYOK model available to this key. |
| `messages` | Required. 1 to 256 messages with roles `user` and `assistant`. |
| `max_tokens` | Required. At most 16,384. |
| `system` | String or array of `text` blocks. |
| `stream` | Anthropic server-sent events. |
| `temperature` | 0 to 1. |
| `top_p` | 0 to 1. |
| `top_k` | 0 to 1000. |
| `stop_sequences` | Up to 4. |
| `metadata` | Only `user_id` is accepted; it maps to the end-user dimension. |
| `tools`, `tool_choice` | Tool definitions with `name`, `description` and `input_schema`. `tool_choice` accepts `auto`, `any`, `tool`, `none` or an object. |

Content blocks of type `text`, `tool_use` and `tool_result` are supported. `image` blocks are rejected.

OpenAI-specific fields (`frequency_penalty`, `presence_penalty`, `n`, `seed`, `response_format`, `user`, `stream_options`, `parallel_tool_calls`) are rejected on the Messages endpoint.

## Retries and timeouts

Inference is **not idempotent**. An SDK retry after a timeout or disconnect can dispatch a second billed request, and a timeout is not evidence that the first request did no provider work. The examples on this page set `max_retries=0`; if your application retries, do so deliberately and only for errors that occurred before dispatch (for example `401`, `403` or `429` with `Retry-After`). See [Errors](/docs/inference/ai-gateway/errors).

Client-side timeouts are a client setting and do not extend the gateway's server-side request lifetime.

## Request size limits

| Limit | Telnyx-hosted models | `gemma-2b-it` | BYOK models |
| --- | --- | --- | --- |
| Output tokens per request | 8,192 | 2,048 | 16,384 |
| Prompt size | About 32k tokens | About 8k tokens | About 128k tokens (about 96k for `gpt-4o` and `gpt-4o-mini`) |

Prompt size is measured conservatively from the request size, so the usable limit can be somewhat lower than the model's tokenizer would count. A request over either limit returns `400`.

Set `max_tokens` (or `max_completion_tokens`) on every request, especially when the key, user or group has a low `tpm_limit`. A request without it reserves the model's full output allowance against budgets and `tpm_limit`, and can be rate-limited even when the actual response would be short. See [Reservations](/docs/inference/ai-gateway/controls#reservations).

## Supported request fields

Unknown top-level fields are rejected with `400`. Options that a specific model does not support are also rejected with `400` rather than silently ignored.

### Chat Completions

| Field | Notes |
| --- | --- |
| `model` | Required. Must be a model returned by `GET /v1/models` for this key. |
| `messages` | Required. Roles: `system`, `developer`, `user`, `assistant`, `tool`. Content is a string or an array of `text` and `image_url` parts. |
| `max_tokens` / `max_completion_tokens` | Send one or the other, never both. At most 8,192 for Telnyx-hosted models (2,048 for `gemma-2b-it`) and 16,384 for BYOK models. Strongly recommended; see [Request size limits](#request-size-limits). |
| `stream`, `stream_options` | Server-sent events. |
| `temperature` | 0 to 2. |
| `top_p` | 0 to 1. |
| `stop` | String or up to 4 strings. |
| `seed`, `presence_penalty`, `frequency_penalty`, `n` | Passed through where the model supports them. |
| `user` | End-user identifier for budgeting and reporting. |
| `tools`, `tool_choice`, `parallel_tool_calls` | Function tools. `tool_choice` accepts `none`, `auto`, `required` or `{"type": "function", "function": {"name": ...}}`. |
| `response_format` | Structured output where the model supports it. |

Image parts must be inline `data:image/...;base64,...` URLs (PNG, JPEG, GIF or WebP). Remote image URLs are rejected.

### Not supported

- Custom provider URLs, provider credentials, routing or fallback controls in the request body.
- Remote image URLs.
- Unlisted beta headers and options.
- The Responses API, embeddings, image and audio generation, batches and realtime APIs. These are not part of the AI Gateway surface; see the [Telnyx Inference API](/docs/inference/getting-started) for those capabilities.

---

### Management API

> Source: https://developers.telnyx.com/docs/inference/ai-gateway/management-api.md

The management plane lives at `https://api.telnyx.com/v2/llm_token_gateway` and authenticates with a Telnyx account API key:

```text
Authorization: Bearer $TELNYX_API_KEY
```

All paths on this page are relative to that base. Every resource is scoped to the authenticated account; a resource that belongs to another account returns `404`.

## Conventions

### Response envelope

Success bodies wrap the resource in `data`. List responses add `meta` with pagination state. Every response carries an `X-Request-ID` correlation header; keep it when reporting a problem. Responses are `Cache-Control: no-store`.

### Idempotency

Every `POST`, `PATCH`, `PUT` and `DELETE` requires an `Idempotency-Key` header. Keys are scoped to account, method and path and retained for 24 hours.

- The same key with the same body returns the original outcome.
- The same key with a different body returns `409` with code `idempotency_conflict`.
- A key whose original request is still in progress returns `409` with `Retry-After`.
- A replayed token-key create returns `200` with metadata only; the secret is never redisclosed. A provider-key secret is never returned, including on replay.

When a mutation times out, retry it with the **same** key and body. Generating a fresh key starts a new operation and can create a duplicate resource.

### ETag preconditions

Every resource carries an integer `version`, returned as a quoted `ETag` header on `GET`, create and update responses. `PATCH` and `DELETE` require that value in `If-Match`:

| Condition | Response |
| --- | --- |
| `If-Match` matches the current version | The mutation is applied. |
| `If-Match` is stale | `412` with code `precondition_failed`. Read the resource again. |
| `If-Match` is missing | `428` with code `precondition_required`. |

End-user caps use `PUT` as a full replacement: send `If-None-Match: *` to create and the current `If-Match` to replace.

### PATCH semantics

- A field omitted from a `PATCH` body is preserved.
- A nullable field set to `null` is cleared (for example `max_budget: null` removes a group or user cap). On a token key, a null limit is reset to its maximum instead; see [Token keys](#token-keys).
- Policy changes apply to new admissions. They do not erase spend history or unresolved exposure, and a request already admitted under the old policy may finish.
- A change can take a short time to propagate. During that window the management response is already committed, but inference may return `503` with code `enforcement_unavailable`. Do not treat a pending change as permission to rely on the old policy.

### Pagination

List endpoints accept `page[number]`, `page[size]` (1 to 100) and `page[snapshot]`.

```json
{
  "data": [ ... ],
  "meta": {
    "page_number": 1,
    "page_size": 100,
    "has_more": true,
    "snapshot": "..."
  }
}
```

The first page returns a `meta.snapshot` bound to the account and filters and valid for 15 minutes. To fetch later pages, increment `page[number]` and pass `page[snapshot]=<meta.snapshot>` with the **same filters**. A missing, expired or mismatched snapshot returns `409`.

## Token groups

A group defines the model allowlist and shared limits for the keys inside it.

| Operation | Path |
| --- | --- |
| Create | `POST /token_groups` |
| List | `GET /token_groups` |
| Read | `GET /token_groups/{id}` |
| Update | `PATCH /token_groups/{id}` |
| Delete | `DELETE /token_groups/{id}` |

| Field | Type | Notes |
| --- | --- | --- |
| `name` | string | Required. 1 to 256 characters. |
| `allowed_models` | string[] | Required. Up to 1000 unique [model names](/docs/inference/ai-gateway/inference-api#models). An empty list permits no inference. |
| `max_budget` | number or null | USD, at most six decimal places. Null is uncapped; zero denies paid requests. |
| `budget_duration` | `1d`, `7d`, `30d` or null | Anchored budget period. Null means a lifetime budget. |
| `rpm_limit`, `tpm_limit` | integer or null | Requests and tokens per rolling 60-second window. Null removes the limit; zero denies. |
| `provider_key_ids` | string[] | [Provider keys](#provider-keys) for [BYOK models](/docs/inference/ai-gateway/inference-api#bring-your-own-key-models). Defaults to `[]`. At most one key per provider; each key must belong to the same account (otherwise `404`) and its provider must match at least one BYOK model in `allowed_models` (otherwise `400`). Two keys for the same provider return `409`. |
| `blocked` | boolean | Blocks every key in the group. |

Read-only fields on the response: `id`, `version`, `spend`, `reserved_spend`, `budget_started_at`, `resets_at`, `created_at`, `updated_at`.

Deleting a group revokes its keys and removes user memberships; spend history is retained.

## Token users

A user represents an application actor that may belong to more than one group. Limits set on the user aggregate across all of its keys in every group.

| Operation | Path |
| --- | --- |
| Create | `POST /token_users` |
| List | `GET /token_users` |
| Read | `GET /token_users/{id}` |
| Update | `PATCH /token_users/{id}` |
| Delete | `DELETE /token_users/{id}` |

| Field | Type | Notes |
| --- | --- | --- |
| `name` | string | Required. |
| `token_group_ids` | string[] | Required. Groups this user may hold keys in. |
| `external_id` | string or null | Your own identifier for the actor. |
| `max_budget`, `budget_duration` | | Aggregate budget across the user's keys. |
| `rpm_limit`, `tpm_limit` | | Aggregate rate limits across the user's keys. |

Removing a group from `token_group_ids` while the user still holds active keys in that group returns `409`; revoke those keys first. Deleting a user revokes its keys and retains spend history.

## Token keys

A key is the credential an application presents to the inference plane.

| Operation | Path |
| --- | --- |
| Create | `POST /token_keys` |
| List | `GET /token_keys` (filters: `token_group_id`, `token_user_id`) |
| Read | `GET /token_keys/{id}` |
| Update | `PATCH /token_keys/{id}` |
| Revoke | `DELETE /token_keys/{id}` |

| Field | Type | Notes |
| --- | --- | --- |
| `name` | string | Required. |
| `token_group_id` | uuid | Required. Cannot be changed after creation. |
| `token_user_id` | uuid or null | Null creates a service key. The user must be a member of the group. Cannot be changed after creation. |
| `allowed_models` | string[] or null | Null inherits the group's list. An empty list denies every model. A non-empty list narrows the group's list. |
| `max_budget` | number or null | USD budget scoped to this key, above 0 and at most 1000, with at most six decimal places. Omitted or null is stored as 1000. |
| `budget_duration` | `1d`, `7d`, `30d` or null | Null means a lifetime budget. |
| `rpm_limit` | integer or null | 1 to 6000. Omitted or null is stored as 6000. |
| `tpm_limit` | integer or null | 1 to 10000000. Omitted or null is stored as 10000000. |
| `expires_at` | date-time or null | After this instant the key is rejected. |
| `blocked` | boolean | Rejects new requests without deleting the key. |
| `required_end_user_id` | boolean | Requires a non-empty `user` / `metadata.user_id` on every request. Presence only, not authenticity. |

Key limits differ from group and user limits: a key is never uncapped and cannot be denied with a zero limit. A value of 0 or above the maximum returns `400` with code `limit_out_of_range`; a negative, fractional or non-numeric value, or a budget with more than six decimal places, returns `400` with code `invalid_request`. On a blocked key, a null limit is kept until the key is unblocked. To deny a key, set `blocked: true` or revoke it. See [Token key limits](/docs/inference/ai-gateway/controls#token-key-limits).

The create response is the **only** place `data.token` appears. It matches `^ltg_sk_[A-Za-z0-9_-]+$`. Store it immediately; a lost token cannot be recovered, only replaced.

`DELETE` revokes the key. Acknowledged revocation blocks new admissions; requests already admitted may complete.

## End users

An end-user cap applies an account-scoped budget or block to a caller-asserted identifier: the value an application sends as OpenAI `user` or Anthropic `metadata.user_id`. The identifier is the resource ID.

| Operation | Path |
| --- | --- |
| List | `GET /end_users` |
| Read | `GET /end_users/{id}` |
| Create or replace | `PUT /end_users/{id}` |
| Delete | `DELETE /end_users/{id}` |

| Field | Type | Notes |
| --- | --- | --- |
| `max_budget` | number or null | Required in the body. Null is uncapped. |
| `budget_duration` | `1d`, `7d`, `30d` or null | Required in the body. |
| `blocked` | boolean | Required in the body. Denies every request carrying this identifier. |

`PUT` is a full replacement and returns `200` for both create and replace. Send `If-None-Match: *` to create and the current `If-Match` to replace; a missing precondition returns `428`.

End-user identifiers are assertions made by whoever holds the token key. Bind them to authenticated users in a trusted backend; see [End-user identity](/docs/inference/ai-gateway/controls#end-user-identity).

## Provider keys

A provider key stores your own OpenAI or Anthropic secret for [bring your own key](/docs/inference/ai-gateway/byok). Attach it to groups through `provider_key_ids`.

| Operation | Path |
| --- | --- |
| Create | `POST /provider_keys` |
| List | `GET /provider_keys` |
| Read | `GET /provider_keys/{id}` |
| Delete | `DELETE /provider_keys/{id}` |

| Field | Type | Notes |
| --- | --- | --- |
| `name` | string | Required. 1 to 256 characters. |
| `provider` | `openai` or `anthropic` | Required. Any other value returns `400`. URLs are never accepted. |
| `secret` | string | Required on create, write-only. Never returned by any response, list or replay. |

Read-only fields on the response: `id`, `version`, `created_at`, `updated_at`.

Provider keys cannot be edited; `PATCH` returns `405`. To change a secret, create a new provider key, attach it to the groups, then delete the old one.

`DELETE` requires the current ETag in `If-Match`, like other deletes. Deleting a provider key detaches it from every group that references it; requests already in progress complete.

## Usage

`GET /spend/events` and `GET /spend/summary` report the requests attributed to these resources. See [Usage reporting](/docs/inference/ai-gateway/usage).

---

## Concepts

### Budgets & rate limits

> Source: https://developers.telnyx.com/docs/inference/ai-gateway/controls.md

Every inference request passes an admission check before it is dispatched to a model. Admission evaluates the token key, its user, its group and the asserted end user together; the request is denied if any scope fails. This page describes each control and its observable behavior.

## Enforcement scopes

| Scope | Set on | Applies to |
| --- | --- | --- |
| Key | `POST /token_keys` | Requests made with that key. |
| User | `POST /token_users` | All keys owned by that user, across every group it belongs to. |
| Group | `POST /token_groups` | All keys in the group. |
| End user | `PUT /end_users/{id}` | Every request in the account that asserts that end-user identifier. |

Model access is the intersection of the group's `allowed_models` and the key's `allowed_models` (when set). A blocked key, user, group or end user denies the request with `403`.

## Budgets

Budgets are USD amounts with at most six decimal places, enforced independently at the key, user, group and end-user scopes.

- On groups, users and end users, `max_budget: null` is uncapped and `max_budget: 0` denies every paid request. Token keys differ; see [Token key limits](#token-key-limits).
- `budget_duration` of `1d`, `7d` or `30d` starts an anchored period at the moment the budget is committed. Periods roll from that anchor, not from midnight or the calendar month. A null duration is a lifetime budget.
- Changing the amount keeps the current period. Changing the duration starts a new anchored period. Neither change erases history.
- The response fields `spend`, `reserved_spend`, `budget_started_at` and `resets_at` on each resource show the current period.

### Reservations

Before a request is dispatched, the gateway reserves a conservative upper bound on its cost from the input size and `max_tokens`. If any scope lacks that much headroom, the request is denied with `403` and code `budget_exceeded` (or `end_user_budget_exceeded`). The reservation is not shrunk to fit.

After the response completes, the reservation is replaced by the actual cost. If the outcome is unknown, for example because the stream was interrupted before usage was reported, the reservation is retained as exposure and the spend event reports `cost: null`. Unknown usage is not zero usage, and a timeout is not a refund.

Budgets are an application control, not an absolute guarantee of spend.

### Budgets and billing

- Usage of Telnyx-hosted models is billed to your Telnyx account at standard [Telnyx AI Inference pricing](https://telnyx.com/pricing/inference-api) for each model.
- Requests on [bring-your-own-key models](/docs/inference/ai-gateway/byok) are billed by your provider on your provider account, not by Telnyx. Budgets and rate limits still apply and act as a guard on that provider spend.
- Budgets and the `cost` values in usage reporting are measured at a flat reference rate of USD 5 per million input tokens and USD 15 per million output tokens, for enforcing limits and attribution. They are not your invoice.

## Rate limits

`rpm_limit` and `tpm_limit` are evaluated over rolling 60-second windows, not calendar minutes, at the key, user and group scopes.

- On groups and users, null removes the limit and zero denies every request. Token keys differ; see [Token key limits](#token-key-limits).
- A rate-limited request returns `429` with a `Retry-After` header.
- `tpm_limit` counts the request's reserved tokens, including the full output allowance when `max_tokens` is not set. Set `max_tokens` to keep requests under a low `tpm_limit`.

## Token key limits

Token keys are always capped. Group and user limits are unchanged: null is uncapped and zero denies.

| Field | Range | Omitted or null |
| --- | --- | --- |
| `max_budget` | Above 0, at most USD 1,000, up to six decimal places | USD 1,000 |
| `rpm_limit` | 1 to 6,000 | 6,000 |
| `tpm_limit` | 1 to 10,000,000 | 10,000,000 |

- An omitted or null key limit is stored as its maximum. On a blocked key, a null limit is kept until the key is unblocked.
- The default USD 1,000 budget is a lifetime budget unless `budget_duration` is set.
- A value of 0 or above the maximum returns `400` with code `limit_out_of_range`.
- A negative, fractional or non-numeric value, or a budget with more than six decimal places, returns `400` with code `invalid_request`.
- To deny a key, block it (`blocked: true`) or revoke it.

## End-user identity

The OpenAI `user` field and the Anthropic Messages `metadata.user_id` field identify an account-scoped end user in the same namespace. The value is used for end-user budgets and blocks and appears as `end_user_id` in usage reporting.

The identifier is an assertion by whoever holds the token key, not an authenticated identity. A key embedded in a client can assert any value. Bind end-user identifiers to authenticated sessions in a trusted backend, and set `required_end_user_id: true` on a key when every request must carry one. That flag checks presence only.

## Mutation safety

Management mutations are protected by idempotency keys and ETag preconditions; see [Conventions](/docs/inference/ai-gateway/management-api#conventions). A committed policy change applies to new admissions once it has propagated. During propagation, inference can return `503` with code `enforcement_unavailable`. Fail closed in that case rather than retrying under the assumption that the previous policy still applies.

---

### Bring your own key

> Source: https://developers.telnyx.com/docs/inference/ai-gateway/byok.md

By default, a token group uses Telnyx-hosted models and usage is billed to your Telnyx account. With bring your own key (BYOK), you store an OpenAI or Anthropic key through the management API and attach it to one or more groups. Requests on [bring-your-own-key models](/docs/inference/ai-gateway/inference-api#bring-your-own-key-models) from those groups run on your provider account, and your provider bills them.

BYOK changes **who is billed**. It does not change attribution, budget enforcement or rate limiting: key, user, group and end-user controls apply exactly as they do for Telnyx-hosted models.

## Set up BYOK

`POST /provider_keys` with a `name`, the `provider` (`openai` or `anthropic`) and the `secret`. The secret is accepted only on create and is never returned by any response, list or idempotent replay.

```bash
curl -X POST https://api.telnyx.com/v2/llm_token_gateway/provider_keys \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "name": "anthropic-production",
    "provider": "anthropic",
    "secret": "<provider-secret>"
  }'
```

```json
{
  "data": {
    "record_type": "provider_key",
    "id": "2d4f6a8b-1c3e-4a5b-9d7f-0e1a2b3c4d5e",
    "name": "anthropic-production",
    "provider": "anthropic",
    "version": 1,
    "created_at": "2026-09-22T10:00:00Z",
    "updated_at": "2026-09-22T10:00:00Z"
  }
}
```

Never keep a logged copy of the request that contains the secret. `GET /provider_keys` lists provider keys and `GET /provider_keys/{id}` reads one; neither shows the secret.

Reference the provider key in the group's `provider_key_ids`, and include the BYOK models you want in `allowed_models`. A group holds at most one key per provider, the key's provider must match at least one BYOK model the group allows, and the key and group must be in the same account.

Set both when creating a group, or update an existing group with an ETag-protected `PATCH`. `allowed_models` is replaced as a whole, so include every model the group should keep:

```bash
curl -X PATCH "https://api.telnyx.com/v2/llm_token_gateway/token_groups/$GROUP_ID" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "If-Match: $GROUP_ETAG" \
  -d '{
    "allowed_models": ["Kimi-K3", "claude-sonnet-5"],
    "provider_key_ids": ["2d4f6a8b-1c3e-4a5b-9d7f-0e1a2b3c4d5e"]
  }'
```

Applications keep using their `ltg_sk_` token key exactly as before. Never place the provider secret in SDK configuration or application code. Call BYOK models on `/v1/chat/completions`; Anthropic BYOK models also work on `/v1/messages` with the [Anthropic SDK](/docs/inference/ai-gateway/inference-api#anthropic-sdk).

A BYOK model works only when the calling key's group has an attached provider key for that model's provider. Without one, requests fail with `503` and code `enforcement_unavailable`. They never fall back to a Telnyx-hosted model.

## Billing and reporting

- Requests on BYOK models are billed by your provider on your provider account, not by Telnyx.
- AI Gateway budgets and rate limits still apply.
- Usage reporting records every BYOK request. Its `cost` is the [budget reference valuation](/docs/inference/ai-gateway/controls#budgets-and-billing), not a Telnyx charge.

## Rotate a provider secret

Provider keys cannot be edited; `PATCH` returns `405`. To change a secret, create a new provider key, attach it to each group in place of the old one, then delete the old provider key.

## Delete a provider key

`DELETE /provider_keys/{id}` requires the provider key's current `ETag` in `If-Match`. Read the key first to get it:

```bash
PROVIDER_KEY_ETAG=$(curl -sS -I "https://api.telnyx.com/v2/llm_token_gateway/provider_keys/$PROVIDER_KEY_ID" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  | awk 'tolower($1) == "etag:" { sub(/\r$/, "", $2); print $2 }')

curl -X DELETE "https://api.telnyx.com/v2/llm_token_gateway/provider_keys/$PROVIDER_KEY_ID" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "If-Match: $PROVIDER_KEY_ETAG"
```

Deleting a provider key detaches it from every group that references it. Requests already in progress complete; new requests on that provider's models from those groups fail with `503` until another key for the provider is attached.

---

### Usage reporting

> Source: https://developers.telnyx.com/docs/inference/ai-gateway/usage.md

Every inference request produces a durable spend event attributed to its token key, token user, token group and asserted end user. Two management endpoints expose that ledger:

| Endpoint | Returns |
| --- | --- |
| `GET /spend/events` | One row per request. |
| `GET /spend/summary` | Totals grouped by one dimension. |

Both live under `https://api.telnyx.com/v2/llm_token_gateway`, authenticate with the Telnyx account API key and use the same date range, filters and pagination.

## Date range and filters

| Parameter | Notes |
| --- | --- |
| `start_date` | Inclusive UTC date, `YYYY-MM-DD`. Required for `/spend/summary`. |
| `end_date` | Exclusive UTC date, `YYYY-MM-DD`. Required for `/spend/summary`. |
| `token_group_id`, `token_user_id`, `token_key_id` | Optional UUID filters. |
| `end_user_id` | Optional end-user identifier filter. |
| `group_by` | `/spend/summary` only, required: `token_group`, `token_user`, `token_key` or `end_user`. |
| `page[number]`, `page[size]`, `page[snapshot]` | Snapshot pagination; see below. |

The range is half-open, `[start_date, end_date)`, and spans at most 31 days. Send ISO dates, not timestamps. If you omit both dates on `/spend/events`, it returns the previous full UTC day. To include today, set `end_date` to tomorrow's date.

## Spend events

```bash
curl --globoff -G https://api.telnyx.com/v2/llm_token_gateway/spend/events \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  --data-urlencode "start_date=2026-09-22" \
  --data-urlencode "end_date=2026-09-23" \
  --data-urlencode "token_group_id=$GROUP_ID" \
  --data-urlencode "page[size]=100"
```

Each event carries:

| Field | Meaning |
| --- | --- |
| `id`, `request_id`, `created_at` | Event identity and the `X-Request-ID` of the inference request. |
| `token_group_id`, `token_user_id`, `token_key_id`, `end_user_id` | Attribution. `token_user_id` is null for service keys; `end_user_id` is null when the request asserted none. |
| `model` | The model requested. |
| `input_tokens`, `output_tokens` | Null when usage is unknown. |
| `cost` | USD at the flat reference rate used for budgets; see [Budgets and billing](/docs/inference/ai-gateway/controls#budgets-and-billing). Null when usage is unknown. |
| `status` | `succeeded`, `failed`, `partial` or `unknown`. |
| `usage_status` | `known`, `unknown` or `reconciled` after a later correction. |
| `configuration_version`, `rate_version` | The policy and rate snapshot the request was valued under. |
| `reservation_micro_usd` | The reservation held for the request in micro-USD. It stays outstanding while usage is unknown. |

## Spend summary

`group_by` selects the dimension. Each row totals the requests attributed to one value of that dimension within the range and filters.

```bash
curl --globoff -G https://api.telnyx.com/v2/llm_token_gateway/spend/summary \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  --data-urlencode "start_date=2026-09-01" \
  --data-urlencode "end_date=2026-10-01" \
  --data-urlencode "group_by=token_key"
```

```json
{
  "data": [
    {
      "dimension_id": "9b7e4a10-3c2d-4f5e-8a6b-1d2c3e4f5a60",
      "name": "support-backend",
      "token_group_id": "5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33",
      "token_user_id": null,
      "token_key_id": "9b7e4a10-3c2d-4f5e-8a6b-1d2c3e4f5a60",
      "end_user_id": null,
      "spend": 1.204311,
      "requests": 4180,
      "input_tokens": 912340,
      "output_tokens": 401277,
      "unknown_requests": 2,
      "reserved_spend": 0.0125
    }
  ],
  "meta": {
    "page_number": 1,
    "page_size": 100,
    "has_more": false,
    "snapshot": "...",
    "start_date": "2026-09-01",
    "end_date": "2026-10-01",
    "group_by": "token_key"
  }
}
```

The four `group_by` dimensions are alternative views of the **same** requests. Do not add totals from different dimensions together.

## Pagination

The first page returns `meta.snapshot`, a stable view bound to the account, range and filters for 15 minutes. While `meta.has_more` is true, request the next `page[number]` with the same parameters plus `page[snapshot]=<meta.snapshot>`. Changing filters or mixing snapshots returns `409`. Treat an export as complete only once `has_more` is false.

## Interpreting the numbers

- **Unknown usage is not zero.** A request whose usage never arrived, for example an interrupted stream, keeps its reservation as `reserved_spend` and reports `cost: null` with `usage_status: "unknown"`. Later reconciliation updates the same event to `reconciled`.
- **Usage is billed at standard pricing.** Usage of Telnyx-hosted models is billed to your Telnyx account at standard [Telnyx AI Inference pricing](https://telnyx.com/pricing/inference-api) for each model.
- **BYOK requests are reported like any other.** Requests on [bring-your-own-key models](/docs/inference/ai-gateway/byok) appear in spend events and summaries like any other request. Their `cost` is the budget reference valuation, not a Telnyx charge; your provider bills them.
- **Spend is not an invoice.** `cost` and `spend` are measured at a flat reference rate (USD 5 per million input tokens, USD 15 per million output tokens) for enforcing budgets and attribution. They are not your invoice.
- **Revoked and deleted resources keep their history.** Filters by ID continue to work after a key, user or group is deleted.

---

### Errors

> Source: https://developers.telnyx.com/docs/inference/ai-gateway/errors.md

Both planes return a structured `errors` array. The inference plane additionally wraps it in the envelope the calling SDK expects, so OpenAI and Anthropic SDK exceptions work unchanged while the Telnyx detail remains available.

## Error envelopes

```json Management
{
  "errors": [
    {
      "code": "precondition_failed",
      "title": "Stale If-Match",
      "detail": "The resource version has changed; read it again.",
      "meta": { "current_version": 4 }
    }
  ]
}
```

```json OpenAI-compatible
{
  "error": {
    "message": "Send either max_tokens or max_completion_tokens, not both.",
    "type": "invalid_request_error",
    "param": "max_completion_tokens",
    "code": "invalid_request"
  },
  "errors": [
    {
      "code": "invalid_request",
      "title": "Invalid request",
      "detail": "Send either max_tokens or max_completion_tokens, not both.",
      "meta": {}
    }
  ]
}
```

```json Anthropic-compatible
{
  "type": "error",
  "error": {
    "type": "permission_error",
    "message": "Model is not available to this token key."
  },
  "request_id": "0b1c2d3e-4f50-4617-8a29-3b4c5d6e7f80",
  "errors": [
    {
      "code": "model_not_in_catalog",
      "title": "Model not allowed",
      "detail": "Model is not available to this token key.",
      "meta": { "scope": "token_key" }
    }
  ]
}
```

Every response carries an `X-Request-ID` header. Keep it, together with the `code`, when reporting a problem. Do not log authorization headers, token keys or full request objects.

An error that occurs after a streaming response has started cannot change the HTTP status. On the OpenAI surface it arrives as an error chunk; on the Anthropic surface, as an `event: error` frame. Consume every stream to completion and handle the SDK's stream exceptions.

## Status codes

| Status | Meaning | Action |
| --- | --- | --- |
| `400` | Invalid body, unsupported option or model-specific option, unknown field, request over the size limits, token key limit out of range, invalid date or page parameter. | Correct the request. |
| `401` | Missing or wrong credential for this plane. | Use a Telnyx API key on the management plane and an `ltg_sk_` token key on the inference plane. |
| `403` | Blocked, revoked or expired key; blocked resource; model not allowed; budget or end-user policy denied. | Read the `code`. Do not treat `403` as retryable. |
| `404` | Resource does not exist or belongs to another account. | Check the ID. |
| `405` | Method not allowed, for example `PATCH` on a provider key. | Provider keys cannot be edited; create a new one instead. |
| `409` | Idempotency conflict, membership conflict or pagination snapshot conflict. | Read the `code`. |
| `412` | Stale `If-Match`. | Read the resource and retry with the current ETag. |
| `428` | Missing `If-Match` (or `If-None-Match: *` on end-user create). | Add the precondition header. |
| `429` | Rate limit exceeded. | Wait for `Retry-After`. Remember that inference retries are not idempotent. |
| `502` | The model provider failed. | Fail closed. The request may or may not have consumed provider work. |
| `503` | Enforcement, catalog, policy propagation or a required dependency is unavailable, or a BYOK model has no attached provider key for its provider. | Fail closed. Do not switch credentials or bypass the gateway. |

## Error codes

| Code | Status | Meaning |
| --- | --- | --- |
| `invalid_request` | 400 | Malformed or unsupported request, including a request over the [size limits](/docs/inference/ai-gateway/inference-api#request-size-limits) and a request to `/v1/messages` with a model that is not an Anthropic BYOK model (use `/v1/chat/completions` for those). |
| `limit_out_of_range` | 400 | A token key `max_budget`, `rpm_limit` or `tpm_limit` is 0 or above its maximum. See [Token key limits](/docs/inference/ai-gateway/controls#token-key-limits). |
| `invalid_token_key` | 401 | The inference credential is missing, malformed or unknown. |
| `unauthorized` | 401 / 403 | The credential is not valid for this plane or resource. |
| `token_key_blocked` | 403 | The key is blocked, revoked or expired. |
| `resource_blocked` | 403 | The key's user, group or asserted end user is blocked. |
| `budget_exceeded` | 403 | A key, user or group budget lacks headroom for the request's reservation. |
| `end_user_budget_exceeded` | 403 | The asserted end user's budget lacks headroom. |
| `model_not_in_catalog` | 403 | The model is not in the key's or group's allowlist. |
| `rate_limit_exceeded` | 429 | An RPM or TPM limit was hit. |
| `not_found` | 404 | No such resource in this account. |
| `conflict` | 409 | Membership or pagination snapshot conflict. |
| `idempotency_conflict` | 409 | The `Idempotency-Key` was reused with a different body, or the original request is still in progress. |
| `precondition_failed` | 412 | `If-Match` does not match the current version. |
| `precondition_required` | 428 | A required precondition header is missing. |
| `enforcement_unavailable` | 503 | Policy, catalog or a dependency needed to admit the request is unavailable, or the request uses a [BYOK model](/docs/inference/ai-gateway/byok) and the key's group has no attached provider key for that model's provider. |
| `upstream_error` | 502 | The model provider returned an error. |

## Handling guidance

- **Never resolve an error by escalating credentials.** A Telnyx API key, provider secret or any other credential is rejected on the inference plane by design.
- **Retry management mutations with the same idempotency key.** A new key on retry can create a duplicate resource.
- **Do not automatically retry inference.** A timeout or `502` is not proof that no provider work happened. Retry only errors that occurred before dispatch, and honor `Retry-After` on `429`.
- **Re-read before re-writing.** On `412`, fetch the resource, review the change that landed, and apply your update to the current version.
- **Fail closed on `503`.** The old policy may no longer apply; wait and retry rather than assuming the request is authorized. For a BYOK model, check that the group has a provider key attached for that model's provider.

---