AI Gateway Guardrails

Guardrails inspect supported text fields in prompts and model responses. Configure a policy on a token group; every token key in that group uses it. Policies apply to new admissions. Existing groups have guardrails: null, which disables inspection. Guardrails detect supported credential and sensitive-data patterns. They do not redact text or provide model-based content moderation, prompt-injection detection, or a guarantee that all sensitive data will be detected.

Configure a policy

Include guardrails when creating a token group, or send it in PATCH /v2/llm_token_gateway/token_groups/{id}. Authenticate with a Telnyx account API key and supply the idempotency and ETag headers required for the mutation. This PATCH body blocks supported secrets in both directions and flags financial data:

{
  "guardrails": {
    "secrets": {
      "prompt": "block",
      "response": "block"
    },
    "dlp": {
      "profiles": ["financial"],
      "prompt": "flag",
      "response": "flag"
    },
    "streaming": "buffered"
  }
}

Each detector has independent prompt and response actions:

Action	Behavior
`ignore`	Do not run the detector at that stage. This is the default.
`flag`	Allow the request or response and record findings.
`block`	Reject a matching prompt or withhold a matching response.

If several rules match, block takes precedence over flag. DLP needs at least one profile whenever either DLP action is flag or block. Unknown fields, duplicate profiles and unsupported values return 400. Omitting guardrails from a PATCH preserves the policy. Supplying a guardrails object replaces the entire nested policy: omitted actions become ignore, and omitted DLP profiles become an empty list. Send the complete intended policy when changing one action. To disable inspection, send:

{
  "guardrails": null
}

A policy update can briefly cause inference to return 503 while the change propagates. Read the group back to verify the saved policy.

Detectors

Detector or profile	Supported patterns
`secrets`	AWS access key IDs; supported GitHub, Slack, Stripe, OpenAI, Anthropic, Google and Telnyx key patterns; AI Gateway token keys; JWT-shaped tokens; private-key block markers.
DLP `financial`	Luhn-valid card-number patterns and checksum-validated IBAN patterns.
DLP `government_id`	US Social Security number patterns and UK National Insurance number patterns.
DLP `contact`	Email address and E.164 phone-number patterns.

These are pattern checks, not verification that a credential is active or an identifier belongs to a real person. Formats outside the supported patterns can be missed; text that resembles a supported format can match. Names, addresses and other free-text personal information are not detected. The gateway inspects supported message text, system text, tool definitions and tool inputs/arguments. Response inspection includes supported text, refusal, thinking and tool-use fields. Opaque provider signatures, encrypted thinking data, provider metadata and unrecognized content-block types are outside this inspection coverage. Guardrails do not inspect images or audio.

Blocking and usage

Stage	Observable result	Usage behavior
Prompt	HTTP `400`, code `prompt_blocked`.	No provider request or budget reservation; no spend event is created.
Complete response	HTTP `400`, code `response_blocked`.	The provider already generated the response. Its usage is accounted for even though the response is withheld.
Buffered streamed response	An SSE error carrying `response_blocked`, after the HTTP `200` headers. No provider content is released.	Provider usage is accounted for.

Guardrail findings are separate from the usage ledger. Do not interpret a blocked response as a refund or automatically retry it. When a cached response is available, it is checked against the current response policy before replay; a blocked cache hit makes no provider call and creates no spend event.

Streaming

streaming accepts buffered or passthrough:

Buffered: the gateway holds the stream until inspection completes, then replays its SSE events. This delays the first content event. Any response block action requires this mode; it is selected automatically if streaming is omitted. Explicit passthrough with a response block action is rejected with 400.
Passthrough: events are forwarded immediately and findings are evaluated after the stream completes. This mode cannot block responses. It is the default when no response action is block.

For Chat Completions, a streamed block is a data: event containing the error document. For Anthropic Messages, it is an event: error frame. Handle stream errors even when the initial HTTP status was 200.

Flagged-response header

Successful responses with flagged findings can include x-ltg-policy. Its value is JSON containing detector codes, counts and actions:

{
  "outcome": "flagged",
  "findings": [
    {
      "detector": "dlp",
      "code": "email",
      "count": 1,
      "action": "flag"
    }
  ]
}

Prompt findings are available on successful complete and streamed responses. Response findings are available in the header only for non-streamed responses; streamed response findings are recorded after the headers have been sent. Blocked findings are available through the events endpoint, not this header. Finding counts represent pattern matches during inspection, not unique sensitive values. Structured data can be inspected in both its JSON spelling and unescaped form, so a value can contribute more than one match.

Query findings

Use GET https://api.telnyx.com/v2/llm_token_gateway/guardrail_events with the Telnyx account API key. Only that account’s events are returned, newest first. Events identify the request, token key, group, optional user/end user, model, stage, outcome and findings. Findings contain codes and counts, never matched text or prompt/response excerpts.

Query parameter	Values or behavior
`start_date`, `end_date`	Supply both as `YYYY-MM-DD`. Start is inclusive and end exclusive, in UTC; the range must be positive and at most 31 days. With neither supplied, the range covers today and the previous six UTC dates.
`token_group_id`, `token_key_id`	Filter by UUID. An unknown or foreign-account ID returns no matching events.
`end_user_id`	Filter by the asserted end-user identifier.
`stage`	`prompt` or `response`.
`outcome`	`flagged`, `blocked`, `evaluated` or `unevaluated`. The current deterministic detectors emit `flagged` and `blocked`; clean passes create no event.
`page[number]`	Positive page number; defaults to `1`.
`page[size]`	`1` to `100`; defaults to `20`.

The response contains data and meta.page_number, meta.page_size, meta.has_more. Unlike resource and spend listings, guardrail events use offset pagination and do not accept page[snapshot]. Newly inserted events can shift page boundaries; deduplicate by event id when collecting several pages. A findings-storage failure does not make a blocked request pass, but it can leave no event for that decision. Use the inference error and X-Request-ID as well when investigating a blocked request.

AI Gateway

API

Concepts

For AI Agents

AI Gateway Guardrails

Configure a policy

Detectors

Blocking and usage

Streaming

Flagged-response header

Query findings

​Configure a policy

​Detectors

​Blocking and usage

​Streaming

​Flagged-response header

​Query findings

Configure a policy

Detectors

Blocking and usage

Streaming

Flagged-response header

Query findings