Default limits by tier
Current defaults as of October 12, 2026. Models with the same default limits are grouped into three rate-limit tiers. Values are tokens per minute, applied per organization, per model, per region.Models by tier
Each model has its own token counters. Models in the same rate-limit tier do
not share a token pool. These groupings are separate from the Flex, Default,
and Priority service tiers.
Organization-specific limits can differ from these defaults. Additional
organization-wide or service-tier limits can also apply; every applicable
limit must have remaining capacity. Selecting a different
service tier does not bypass a model-wide or
organization-wide limit.
These tables describe token limits, not model availability or guaranteed
throughput. Check Get available models
for the current catalog. A model omitted from this table is not necessarily
unlimited; contact Telnyx Support for its limits.
How tokens count
The budgets are checked independently. Cached input still consumes the total
input budget. Uncached input consumes both input budgets.
For example, a completed request with 10,000 prompt tokens, including 8,000
cached tokens, and 1,000 completion tokens consumes 10,000 total input,
2,000 uncached input, and 1,000 output tokens.
Limits use fixed UTC calendar minutes and reset at the next minute boundary.
Each region counts independently. Usage settles in the minute a request
completes; an input estimate may temporarily reserve total-input capacity
while the request is running. Streaming output therefore appears in the
counter after completion, and reported headroom can lag concurrent usage.
These TPM limits apply to billable public
Chat Completions,
Responses,
and Anthropic Messages requests. They do not apply
to AI Assistants’ internal LLM requests, classification, or embeddings. Requests
to third-party providers using the customer’s provider key (BYOK), and
customer-supplied model endpoints, are not counted by this limiter.
TPM limits are separate from a model’s per-request context window and from
inference spending limits, which cap spend
in USD and also cover voice-assistant inference.
Read remaining capacity
When token limits are enforced for a request, response headers describe the applicable limits and remaining tokens. For example:limit/remaining pair reports the applicable scope with the least
remaining capacity for that token type. An organization-wide limit can
therefore appear instead of the model default. x-ratelimit-reset-tokens is
a duration until the minute resets, not a timestamp.
Headers are a snapshot when the request is checked. For an admitted request,
they include its input reservation but not its eventual output. They are also
returned on streaming responses and token-limit rejections. A missing header
does not establish that usage is unlimited.
Handle a token-limit response
When any applicable token counter has reached its limit, new requests receive HTTP429 Too Many Requests. The response identifies the exceeded token
budget and scope. Retry-After gives the seconds until the minute resets
(1–60); x-ratelimit-reset-tokens expresses the same delay as a duration.
The Anthropic-compatible endpoint uses the rate_limit_error error type.
Requests rejected by the token limiter do not run the model and are not billed.
Requests already running continue. Concurrent or large requests can cause
usage to exceed a limit before subsequent requests are rejected.
- Pause new requests within the scope identified by the response: organization, model, service tier, or model plus service tier. Small/Medium/Large groupings do not share a token pool.
- Wait at least
Retry-Afterseconds before retrying. Add jitter to avoid sending all queued requests at once, and bound retry attempts. - Reduce concurrency, prompt size, or requested output as appropriate for the exhausted budget. Prompt caching reduces uncached input usage but still consumes total input capacity.