Skip to main content
Every inference request passes an admission check before it is dispatched to a model. Admission evaluates the token key, its user, its group and the asserted end user together; the request is denied if any scope fails. This page describes each control and its observable behavior.

​
Enforcement scopes

Model access is the intersection of the group’s allowed_models and the key’s allowed_models (when set). A blocked key, user, group or end user denies the request with 403.

​
Budgets

Budgets are USD amounts with at most six decimal places, enforced independently at the key, user, group and end-user scopes.
  • On groups, users and end users, max_budget: null is uncapped and max_budget: 0 denies every paid request. Token keys differ; see Token key limits.
  • budget_duration of 1d, 7d or 30d starts an anchored period at the moment the budget is committed. Periods roll from that anchor, not from midnight or the calendar month. A null duration is a lifetime budget.
  • Changing the amount keeps the current period. Changing the duration starts a new anchored period. Neither change erases history.
  • The response fields spend, reserved_spend, budget_started_at and resets_at on each resource show the current period.

​
Reservations

Before a request is dispatched, the gateway reserves a conservative upper bound on its cost from the input size and max_tokens. If any scope lacks that much headroom, the request is denied with 403 and code budget_exceeded (or end_user_budget_exceeded). The reservation is not shrunk to fit. After the response completes, the reservation is replaced by the actual cost. If the outcome is unknown, for example because the stream was interrupted before usage was reported, the reservation is retained as exposure and the spend event reports cost: null. Unknown usage is not zero usage, and a timeout is not a refund. Budgets are an application control, not an absolute guarantee of spend.

​
Budgets and billing

  • Usage of Telnyx-hosted models is billed to your Telnyx account at standard Telnyx AI Inference pricing for each model.
  • Requests on bring-your-own-key models are billed by your provider on your provider account, not by Telnyx. Budgets and rate limits still apply and act as a guard on that provider spend.
  • Budgets and the cost values in usage reporting are measured at a flat reference rate of USD 5 per million input tokens and USD 15 per million output tokens, for enforcing limits and attribution. They are not your invoice.

​
Rate limits

rpm_limit and tpm_limit are evaluated over rolling 60-second windows, not calendar minutes, at the key, user and group scopes.
  • On groups and users, null removes the limit and zero denies every request. Token keys differ; see Token key limits.
  • A rate-limited request returns 429 with a Retry-After header.
  • tpm_limit counts the request’s reserved tokens, including the full output allowance when max_tokens is not set. Set max_tokens to keep requests under a low tpm_limit.

​
Token key limits

Token keys are always capped. Group and user limits are unchanged: null is uncapped and zero denies.
  • An omitted or null key limit is stored as its maximum. On a blocked key, a null limit is kept until the key is unblocked.
  • The default USD 1,000 budget is a lifetime budget unless budget_duration is set.
  • A value of 0 or above the maximum returns 400 with code limit_out_of_range.
  • A negative, fractional or non-numeric value, or a budget with more than six decimal places, returns 400 with code invalid_request.
  • To deny a key, block it (blocked: true) or revoke it.

​
End-user identity

The OpenAI user field and the Anthropic Messages metadata.user_id field identify an account-scoped end user in the same namespace. The value is used for end-user budgets and blocks and appears as end_user_id in usage reporting. The identifier is an assertion by whoever holds the token key, not an authenticated identity. A key embedded in a client can assert any value. Bind end-user identifiers to authenticated sessions in a trusted backend, and set required_end_user_id: true on a key when every request must carry one. That flag checks presence only.

​
Mutation safety

Management mutations are protected by idempotency keys and ETag preconditions; see Conventions. A committed policy change applies to new admissions once it has propagated. During propagation, inference can return 503 with code enforcement_unavailable. Fail closed in that case rather than retrying under the assumption that the previous policy still applies.