AI Gateway Management API

The management plane lives at https://api.telnyx.com/v2/llm_token_gateway and authenticates with a Telnyx account API key:

Authorization: Bearer $TELNYX_API_KEY

All paths on this page are relative to that base. Every resource is scoped to the authenticated account; a resource that belongs to another account returns 404.

Conventions

Response envelope

Success bodies wrap the resource in data. List responses add meta with pagination state. Every response carries an X-Request-ID correlation header; keep it when reporting a problem. Responses are Cache-Control: no-store.

Idempotency

Every POST, PATCH, PUT and DELETE requires an Idempotency-Key header. Keys are scoped to account, method and path and retained for 24 hours.

The same key with the same body returns the original outcome.
The same key with a different body returns 409 with code idempotency_conflict.
A key whose original request is still in progress returns 409 with Retry-After.
A replayed token-key create returns 200 with metadata only; the secret is never redisclosed. A provider-key secret is never returned, including on replay.

When a mutation times out, retry it with the same key and body. Generating a fresh key starts a new operation and can create a duplicate resource.

ETag preconditions

Every resource carries an integer version, returned as a quoted ETag header on GET, create and update responses. PATCH and DELETE require that value in If-Match:

Condition	Response
`If-Match` matches the current version	The mutation is applied.
`If-Match` is stale	`412` with code `precondition_failed`. Read the resource again.
`If-Match` is missing	`428` with code `precondition_required`.

End-user caps use PUT as a full replacement: send If-None-Match: * to create and the current If-Match to replace.

PATCH semantics

A field omitted from a PATCH body is preserved.
A nullable field set to null is cleared (for example max_budget: null removes a group or user cap). On a token key, a null limit is reset to its maximum instead; see Token keys.
Policy changes apply to new admissions. They do not erase spend history or unresolved exposure, and a request already admitted under the old policy may finish.
A change can take a short time to propagate. During that window the management response is already committed, but inference may return 503 with code enforcement_unavailable. Do not treat a pending change as permission to rely on the old policy.

Pagination

List endpoints accept page[number], page[size] (1 to 100) and page[snapshot].

{
  "data": [ ... ],
  "meta": {
    "page_number": 1,
    "page_size": 100,
    "has_more": true,
    "snapshot": "..."
  }
}

The first page returns a meta.snapshot bound to the account and filters and valid for 15 minutes. To fetch later pages, increment page[number] and pass page[snapshot]=<meta.snapshot> with the same filters. A missing, expired or mismatched snapshot returns 409.

Token groups

A group defines the model allowlist and shared limits for the keys inside it.

Operation	Path
Create	`POST /token_groups`
List	`GET /token_groups`
Read	`GET /token_groups/{id}`
Update	`PATCH /token_groups/{id}`
Delete	`DELETE /token_groups/{id}`

Field	Type	Notes
`name`	string	Required. 1 to 256 characters.
`allowed_models`	string[]	Required. Up to 1000 unique model names. An empty list permits no inference.
`max_budget`	number or null	USD, at most six decimal places. Null is uncapped; zero denies paid requests.
`budget_duration`	`1d`, `7d`, `30d` or null	Anchored budget period. Null means a lifetime budget.
`rpm_limit`, `tpm_limit`	integer or null	Requests and tokens per rolling 60-second window. Null removes the limit; zero denies.
`provider_key_ids`	string[]	Provider keys for BYOK models. Defaults to `[]`. At most one key per provider; each key must belong to the same account (otherwise `404`) and its provider must match at least one BYOK model in `allowed_models` (otherwise `400`). Two keys for the same provider return `409`.
`blocked`	boolean	Blocks every key in the group.

Read-only fields on the response: id, version, spend, reserved_spend, budget_started_at, resets_at, created_at, updated_at. Deleting a group revokes its keys and removes user memberships; spend history is retained.

Token users

A user represents an application actor that may belong to more than one group. Limits set on the user aggregate across all of its keys in every group.

Operation	Path
Create	`POST /token_users`
List	`GET /token_users`
Read	`GET /token_users/{id}`
Update	`PATCH /token_users/{id}`
Delete	`DELETE /token_users/{id}`

Field	Type	Notes
`name`	string	Required.
`token_group_ids`	string[]	Required. Groups this user may hold keys in.
`external_id`	string or null	Your own identifier for the actor.
`max_budget`, `budget_duration`		Aggregate budget across the user’s keys.
`rpm_limit`, `tpm_limit`		Aggregate rate limits across the user’s keys.

Removing a group from token_group_ids while the user still holds active keys in that group returns 409; revoke those keys first. Deleting a user revokes its keys and retains spend history.

Token keys

A key is the credential an application presents to the inference plane.

Operation	Path
Create	`POST /token_keys`
List	`GET /token_keys` (filters: `token_group_id`, `token_user_id`)
Read	`GET /token_keys/{id}`
Update	`PATCH /token_keys/{id}`
Revoke	`DELETE /token_keys/{id}`

Field	Type	Notes
`name`	string	Required.
`token_group_id`	uuid	Required. Cannot be changed after creation.
`token_user_id`	uuid or null	Null creates a service key. The user must be a member of the group. Cannot be changed after creation.
`allowed_models`	string[] or null	Null inherits the group’s list. An empty list denies every model. A non-empty list narrows the group’s list.
`max_budget`	number or null	USD budget scoped to this key, above 0 and at most 1000, with at most six decimal places. Omitted or null is stored as 1000.
`budget_duration`	`1d`, `7d`, `30d` or null	Null means a lifetime budget.
`rpm_limit`	integer or null	1 to 6000. Omitted or null is stored as 6000.
`tpm_limit`	integer or null	1 to 10000000. Omitted or null is stored as 10000000.
`expires_at`	date-time or null	After this instant the key is rejected.
`blocked`	boolean	Rejects new requests without deleting the key.
`required_end_user_id`	boolean	Requires a non-empty `user` / `metadata.user_id` on every request. Presence only, not authenticity.

Key limits differ from group and user limits: a key is never uncapped and cannot be denied with a zero limit. A value of 0 or above the maximum returns 400 with code limit_out_of_range; a negative, fractional or non-numeric value, or a budget with more than six decimal places, returns 400 with code invalid_request. On a blocked key, a null limit is kept until the key is unblocked. To deny a key, set blocked: true or revoke it. See Token key limits. The create response is the only place data.token appears. It matches ^ltg_sk_[A-Za-z0-9_-]+$. Store it immediately; a lost token cannot be recovered, only replaced. DELETE revokes the key. Acknowledged revocation blocks new admissions; requests already admitted may complete.

End users

An end-user cap applies an account-scoped budget or block to a caller-asserted identifier: the value an application sends as OpenAI user or Anthropic metadata.user_id. The identifier is the resource ID.

Operation	Path
List	`GET /end_users`
Read	`GET /end_users/{id}`
Create or replace	`PUT /end_users/{id}`
Delete	`DELETE /end_users/{id}`

Field	Type	Notes
`max_budget`	number or null	Required in the body. Null is uncapped.
`budget_duration`	`1d`, `7d`, `30d` or null	Required in the body.
`blocked`	boolean	Required in the body. Denies every request carrying this identifier.

PUT is a full replacement and returns 200 for both create and replace. Send If-None-Match: * to create and the current If-Match to replace; a missing precondition returns 428. End-user identifiers are assertions made by whoever holds the token key. Bind them to authenticated users in a trusted backend; see End-user identity.

Provider keys

A provider key stores your own OpenAI or Anthropic secret for bring your own key. Attach it to groups through provider_key_ids.

Operation	Path
Create	`POST /provider_keys`
List	`GET /provider_keys`
Read	`GET /provider_keys/{id}`
Delete	`DELETE /provider_keys/{id}`

Field	Type	Notes
`name`	string	Required. 1 to 256 characters.
`provider`	`openai` or `anthropic`	Required. Any other value returns `400`. URLs are never accepted.
`secret`	string	Required on create, write-only. Never returned by any response, list or replay.

Read-only fields on the response: id, version, created_at, updated_at. Provider keys cannot be edited; PATCH returns 405. To change a secret, create a new provider key, attach it to the groups, then delete the old one. DELETE requires the current ETag in If-Match, like other deletes. Deleting a provider key detaches it from every group that references it; requests already in progress complete.

Usage

GET /spend/events and GET /spend/summary report the requests attributed to these resources. See Usage reporting.

AI Gateway

API

Concepts

For AI Agents

AI Gateway Management API

Conventions

Response envelope

Idempotency

ETag preconditions

PATCH semantics

Token groups

Token users

Token keys

End users

Provider keys

Usage

​Conventions

​Response envelope

​Idempotency

​ETag preconditions

​PATCH semantics

​Pagination

​Token groups

​Token users

​Token keys

​End users

​Provider keys

​Usage

Conventions

Response envelope

Idempotency

ETag preconditions

PATCH semantics

Pagination

Token groups

Token users

Token keys

End users

Provider keys

Usage