> ## Documentation Index
> Fetch the complete documentation index at: https://developers.telnyx.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference token rate limits

> Current tokens-per-minute limits by inference model, how cached input counts, and how to monitor headroom and retry rate-limited requests.

Telnyx Inference enforces separate tokens-per-minute (TPM) limits for total
input, uncached input, and output. Limits are shared by requests from the same
organization, including requests authenticated with different API keys.
Accounts without an organization use their own account limits.

## Default limits by tier

Current defaults as of **October 12, 2026**. Models with the same default limits
are grouped into three rate-limit tiers. Values are tokens per minute, applied
per organization, per model, per region.

| Tier | Total input TPM | Uncached input TPM | Output TPM |
| - | -: | -: | -: |
| Small | 100,000,000 | 20,000,000 | 1,000,000 |
| Medium | 40,000,000 | 8,000,000 | 400,000 |
| Large | 20,000,000 | 4,000,000 | 200,000 |

### Models by tier

| Tier | Model IDs |
| - | - |
| Small | `zai-org/GLM-5.3-Flash`, `deepseek-ai/DeepSeek-V4.1-Flash`, `google/gemma-4-26B-A4B-it`, `google/gemma-4-31B-it`, `MiniMaxAI/MiniMax-M3-MXFP8`, `Qwen/Qwen3.8-27B` |
| Medium | `zai-org/GLM-5.3`, `zai-org/GLM-5.2`, `moonshotai/Kimi-K2.6` |
| Large | `moonshotai/Kimi-K3` |

Each model has its own token counters. Models in the same rate-limit tier do
not share a token pool. These groupings are separate from the Flex, Default,
and Priority [service tiers](/docs/inference/service-tiers).

Organization-specific limits can differ from these defaults. Additional
organization-wide or service-tier limits can also apply; every applicable
limit must have remaining capacity. Selecting a different
[service tier](/docs/inference/service-tiers) does not bypass a model-wide or
organization-wide limit.

These tables describe token limits, not model availability or guaranteed
throughput. Check [Get available models](/api-reference/openai-chat/get-available-models-openai-compatible)
for the current catalog. A model omitted from this table is not necessarily
unlimited; contact [Telnyx Support](https://support.telnyx.com) for its limits.

## How tokens count

| Budget | Tokens counted |
| - | - |
| Total input | All prompt tokens, including cached prompt tokens |
| Uncached input | Prompt tokens minus cached prompt tokens |
| Output | Generated completion tokens |

The budgets are checked independently. Cached input still consumes the total
input budget. Uncached input consumes both input budgets.

For example, a completed request with 10,000 prompt tokens, including 8,000
cached tokens, and 1,000 completion tokens consumes **10,000 total input**,
**2,000 uncached input**, and **1,000 output** tokens.

Limits use fixed UTC calendar minutes and reset at the next minute boundary.
Each region counts independently. Usage settles in the minute a request
completes; an input estimate may temporarily reserve total-input capacity
while the request is running. Streaming output therefore appears in the
counter after completion, and reported headroom can lag concurrent usage.

These TPM limits apply to billable public
[Chat Completions](/api-reference/openai-chat/create-a-chat-completion-openai-compatible),
[Responses](/api-reference/openai-chat/create-an-openai-compatible-response),
and [Anthropic Messages](/docs/inference/anthropic) requests. They do not apply
to AI Assistants' internal LLM requests, classification, or embeddings. Requests
to third-party providers using the customer's provider key (BYOK), and
customer-supplied model endpoints, are not counted by this limiter.

TPM limits are separate from a model's per-request context window and from
[inference spending limits](/docs/inference/spending-limits), which cap spend
in USD and also cover voice-assistant inference.

## Read remaining capacity

When token limits are enforced for a request, response headers describe the
applicable limits and remaining tokens. For example:

```http theme={null}
x-ratelimit-limit-input-tokens: 40000000
x-ratelimit-remaining-input-tokens: 39990000
x-ratelimit-limit-uncached-input-tokens: 8000000
x-ratelimit-remaining-uncached-input-tokens: 7998000
x-ratelimit-limit-output-tokens: 400000
x-ratelimit-remaining-output-tokens: 399000
x-ratelimit-reset-tokens: 23s
```

Each `limit`/`remaining` pair reports the applicable scope with the least
remaining capacity for that token type. An organization-wide limit can
therefore appear instead of the model default. `x-ratelimit-reset-tokens` is
a duration until the minute resets, not a timestamp.

Headers are a snapshot when the request is checked. For an admitted request,
they include its input reservation but not its eventual output. They are also
returned on streaming responses and token-limit rejections. A missing header
does not establish that usage is unlimited.

## Handle a token-limit response

When any applicable token counter has reached its limit, new requests receive
HTTP `429 Too Many Requests`. The response identifies the exceeded token
budget and scope. `Retry-After` gives the seconds until the minute resets
(1–60); `x-ratelimit-reset-tokens` expresses the same delay as a duration.
The Anthropic-compatible endpoint uses the `rate_limit_error` error type.

Requests rejected by the token limiter do not run the model and are not billed.
Requests already running continue. Concurrent or large requests can cause
usage to exceed a limit before subsequent requests are rejected.

1. Pause new requests within the scope identified by the response: organization,
   model, service tier, or model plus service tier. Small/Medium/Large groupings
   do not share a token pool.
2. Wait at least `Retry-After` seconds before retrying. Add jitter to avoid
   sending all queued requests at once, and bound retry attempts.
3. Reduce concurrency, prompt size, or requested output as appropriate for the
   exhausted budget. Prompt caching reduces uncached input usage but still
   consumes total input capacity.

Other capacity limits can also return HTTP 429. Inspect the response detail
and headers before treating every 429 as an exhausted TPM budget.

## Request higher limits

Contact [Telnyx Support](https://support.telnyx.com) with the organization ID,
model IDs, regions, service tiers, desired total-input/uncached-input/output
TPM, and expected sustained and peak traffic. Include recent rate-limit
headers and request IDs when investigating throttling. Do not include API keys.