AI Gateway Quickstart

This quickstart provisions a group and a service token key through the management API, makes an inference request with that key, reads the resulting usage and revokes the key. Every step is a copy-paste request.

Prerequisites

A Telnyx API key from the portal.
A model name from the available models. This guide uses Kimi-K3. The inference GET /v1/models endpoint is scoped to a token key, so it cannot be used to discover models before a key exists.
curl, jq and uuidgen (or another way to generate a UUID for the Idempotency-Key header).

Export the credential, the model name and the two base URLs so the examples work as-is:

export TELNYX_API_KEY="KEY..."
export AI_GATEWAY_MODEL="Kimi-K3"
export AI_GATEWAY_MANAGEMENT_BASE_URL="https://api.telnyx.com/v2/llm_token_gateway"
export AI_GATEWAY_INFERENCE_BASE_URL="https://llm.telnyx.com/v1"

This walkthrough creates persistent account resources and can incur model charges.

Create a token group

A token group defines which models its keys may call and the limits they share. name and allowed_models are required. An empty allowed_models list permits no inference.This example sets a USD 10 budget per anchored one-day period and 60 requests per minute.

curl -X POST "$AI_GATEWAY_MANAGEMENT_BASE_URL/token_groups" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d "$(jq -n --arg model "$AI_GATEWAY_MODEL" '{
    name: "support-assistant",
    allowed_models: [$model],
    max_budget: 10,
    budget_duration: "1d",
    rpm_limit: 60
  }')"

{
  "data": {
    "record_type": "token_group",
    "id": "5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33",
    "name": "support-assistant",
    "allowed_models": ["Kimi-K3"],
    "max_budget": 10,
    "budget_duration": "1d",
    "rpm_limit": 60,
    "tpm_limit": null,
    "provider_key_ids": [],
    "blocked": false,
    "spend": 0,
    "reserved_spend": 0,
    "budget_started_at": "2026-09-22T10:00:00Z",
    "resets_at": "2026-09-23T10:00:00Z",
    "version": 1,
    "created_at": "2026-09-22T10:00:00Z",
    "updated_at": "2026-09-22T10:00:00Z"
  }
}

Keep data.id; the next step needs it and the group is what you retire at the end.

export GROUP_ID="5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33"

Every mutation requires an Idempotency-Key. If a request times out, retry it with the same key and body; a new key starts a different operation and can create a second resource.

Issue a token key

A token key is the credential the application uses. token_user_id: null creates a service key owned by the group alone. Null allowed_models inherits the group’s list. Key limits that are omitted are set to their maximums: USD 1,000 lifetime budget, 6,000 requests and 10,000,000 tokens per minute. See Token key limits.

curl -X POST "$AI_GATEWAY_MANAGEMENT_BASE_URL/token_keys" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d "$(jq -n --arg group "$GROUP_ID" '{
    name: "support-backend",
    token_group_id: $group,
    token_user_id: null
  }')"

{
  "data": {
    "record_type": "token_key",
    "id": "9b7e4a10-3c2d-4f5e-8a6b-1d2c3e4f5a60",
    "name": "support-backend",
    "token_group_id": "5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33",
    "token_user_id": null,
    "allowed_models": null,
    "max_budget": 1000,
    "budget_duration": null,
    "rpm_limit": 6000,
    "tpm_limit": 10000000,
    "expires_at": null,
    "blocked": false,
    "required_end_user_id": false,
    "spend": 0,
    "reserved_spend": 0,
    "budget_started_at": null,
    "resets_at": null,
    "version": 1,
    "created_at": "2026-09-22T10:01:00Z",
    "updated_at": "2026-09-22T10:01:00Z",
    "token": "ltg_sk_..."
  }
}

data.token is returned only on the original create response. GET, list and idempotent replays return metadata without the secret. If the response is lost, revoke the key and create a new one with a new idempotency key.

Store the token in your secret store now, then export it and the key ID for the remaining steps:

export TOKEN_KEY_ID="9b7e4a10-3c2d-4f5e-8a6b-1d2c3e4f5a60"
export AI_GATEWAY_TOKEN_KEY="ltg_sk_..."

To issue a key for a specific application user instead, first POST /token_users with name and token_group_ids: [GROUP_ID], then pass the returned ID as token_user_id. Key ownership cannot be changed later; issue a replacement key instead.

Discover models and make a request

The inference plane authenticates with the token key, not the account API key. GET /v1/models returns only the models this key may use.

curl "$AI_GATEWAY_INFERENCE_BASE_URL/models" \
  -H "Authorization: Bearer $AI_GATEWAY_TOKEN_KEY"

Send a Chat Completions request with one of those models:

curl -X POST "$AI_GATEWAY_INFERENCE_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_TOKEN_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg model "$AI_GATEWAY_MODEL" '{
    model: $model,
    messages: [{role: "user", content: "Hello"}],
    max_tokens: 32
  }')"

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AI_GATEWAY_TOKEN_KEY"],
    base_url=os.environ["AI_GATEWAY_INFERENCE_BASE_URL"],
    max_retries=0,
)

completion = client.chat.completions.create(
    model=os.environ["AI_GATEWAY_MODEL"],
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=32,
)
print(completion.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_TOKEN_KEY,
  baseURL: process.env.AI_GATEWAY_INFERENCE_BASE_URL,
  maxRetries: 0,
});

const completion = await client.chat.completions.create({
  model: process.env.AI_GATEWAY_MODEL,
  messages: [{ role: "user", content: "Hello" }],
  max_tokens: 32,
});
console.log(completion.choices[0].message.content);

Inference requests are not idempotent. The examples disable SDK retries so that a timeout cannot silently trigger a second billed request. Set max_tokens on every request: a request without it reserves the model’s full output allowance against budgets and tpm_limit. See Inference API for streaming and request size limits.

Inspect usage

Spend events are queried over a half-open UTC date range [start_date, end_date) of at most 31 days, using ISO dates rather than timestamps. Filter by the key you just used:

curl --globoff -G "$AI_GATEWAY_MANAGEMENT_BASE_URL/spend/events" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  --data-urlencode "start_date=2026-09-22" \
  --data-urlencode "end_date=2026-09-23" \
  --data-urlencode "token_key_id=$TOKEN_KEY_ID" \
  --data-urlencode "page[size]=100"

{
  "data": [
    {
      "id": "c0a1b2c3-d4e5-4f60-8a71-92b3c4d5e6f7",
      "request_id": "0b1c2d3e-4f50-4617-8a29-3b4c5d6e7f80",
      "created_at": "2026-09-22T10:02:14Z",
      "token_group_id": "5f1c9d2e-7b3a-4c8e-9f21-0a6d4e8b1c33",
      "token_user_id": null,
      "token_key_id": "9b7e4a10-3c2d-4f5e-8a6b-1d2c3e4f5a60",
      "end_user_id": null,
      "model": "Kimi-K3",
      "input_tokens": 9,
      "output_tokens": 12,
      "cost": 0.000225,
      "status": "succeeded",
      "usage_status": "known",
      "configuration_version": 1,
      "rate_version": "telnyx-reference-v1",
      "reservation_micro_usd": 0
    }
  ],
  "meta": {
    "page_number": 1,
    "page_size": 100,
    "has_more": false,
    "snapshot": "..."
  }
}

When meta.has_more is true, request the next page[number] with the same filters and page[snapshot]=<meta.snapshot>. See Usage reporting for summaries grouped by group, user, key or end user.

Revoke the key

DELETE requires the resource’s current ETag in If-Match. Read the key first; the metadata GET never reveals the token.

KEY_ETAG=$(curl -sS -I "$AI_GATEWAY_MANAGEMENT_BASE_URL/token_keys/$TOKEN_KEY_ID" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  | awk 'tolower($1) == "etag:" { sub(/\r$/, "", $2); print $2 }')

curl -X DELETE "$AI_GATEWAY_MANAGEMENT_BASE_URL/token_keys/$TOKEN_KEY_ID" \
  -H "Authorization: Bearer $TELNYX_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "If-Match: $KEY_ETAG"

A 204 means the key is revoked: new requests with that token are rejected, while requests already admitted may finish. Spend history is retained. On 412, the resource changed since the read; fetch it again rather than forcing the write.The group stays available for new keys. To retire it as well, DELETE /token_groups/{id} with a fresh ETag and idempotency key. Deleting a group cascades to its keys.

Next: use your own provider key

To run OpenAI or Anthropic models on your own provider account, store the provider key once, attach it to a group and allow a bring-your-own-key model. Applications keep using the same token key. See Bring your own key.

Next steps

Inference API

Available models, streaming, request limits and the supported request fields.

Management API

Users, end users, PATCH semantics and pagination.

Budgets and rate limits

What each limit does and how it is enforced.

Errors

Status codes and structured error codes on both planes.

Bring your own key

Attach your own OpenAI or Anthropic key to a group.

AI Gateway

API

Concepts

For AI Agents

AI Gateway Quickstart

Prerequisites

Next: use your own provider key

Next steps

Inference API

Management API

Budgets and rate limits

Errors

Bring your own key

​Prerequisites

​Next: use your own provider key

​Next steps

Inference API

Management API

Budgets and rate limits

Errors

Bring your own key

Prerequisites

Next: use your own provider key

Next steps