Skip to main content

Telnyx Voice — Full Documentation

Voice API, speech-to-text, text-to-speech, and voice design. Complete page content for the Voice section of the Telnyx developer docs (https://developers.telnyx.com). Root index: https://developers.telnyx.com/llms.txt · Lightweight index for this section: https://developers.telnyx.com/development/llms/voice-llms-txt.md

Overview

Overview

Source: https://developers.telnyx.com/docs/voice/overview.md
Real-time and file-based transcription with four engine options: Telnyx, Google, Deepgram, and Azure. Convert text to natural-sounding audio via WebSocket streaming. Multiple voice providers and languages. Clone voices from audio samples or design custom AI voices from natural language prompts.

STT

Overview

Source: https://developers.telnyx.com/docs/voice/stt/overview.md
Telnyx STT transcribes audio to text in three ways: Stream audio over a persistent WebSocket connection. Real-time partial and final transcripts. Upload audio files via REST API. OpenAI-compatible endpoint. Enable transcription during live voice calls via Call Control or TeXML.

Quickstart

Source: https://developers.telnyx.com/docs/voice/stt/getting-started.md

Prerequisites

That’s it. No other setup required. Real-time transcription over a persistent connection. Send audio, get partial and final transcripts back as they happen. Install:
main.py:
Run it:
Install:
index.js:
Run it:

Example output

Upload an audio file and get the full transcript back. The endpoint is OpenAI SDK compatible — swap base_url and api_key and your existing code works. Install:
main.py:
Run it:
Install:
index.js:
Run it:
Or transcribe from a URL (no file upload needed):

Example response

For segment- or word-level timestamps, use model="deepgram/nova-3" with response_format=verbose_json. The Whisper models (openai/whisper-large-v3-turbo, openai/whisper-tiny) return text only.

What’s next


Models

Source: https://developers.telnyx.com/docs/voice/stt/models.md

Comparison

Engine Details

The default WebSocket engine. Best English accuracy and the richest feature set. For REST, you must explicitly set model="deepgram/nova-3" — the REST default is openai/whisper-large-v3-turbo. Models:
  • nova-3 — Latest and most accurate. Supports diarization, word-level timestamps, smart formatting, numerals, and punctuation via model_config. Use this unless you need the lowest possible latency.
  • nova-2 — Previous generation. Still supported but nova-3 is better in all benchmarks.
  • flux — Purpose-built for voice agents. Lowest latency with built-in end-of-turn detection — tells you when the speaker has finished so your agent can respond. WebSocket only.
Languages: 40+ languages across Deepgram models. Nova-3 supports multi mode (10 languages with code-switching). Flux supports English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch. See Deepgram languages. Telnyx runs Whisper models on-network. Models:
  • openai/whisper-large-v3-turbo — Multilingual (50+ languages, auto-detected). Returns text only — no timestamps regardless of response format.
  • openai/whisper-tiny — Lightweight, lowest resource usage.
Languages: 50+ languages, auto-detected. Use auto_detect to skip the language hint. See the Whisper language list. Limitations: No diarization. No word-level timestamps. Google Cloud Speech-to-Text integration. Model: latest_long Languages: 125+ languages/locales. See Google Cloud STT languages. Microsoft Azure Speech Services integration. Model: azure/fast Languages: 100+ languages/locales with strong accent and dialect coverage. See Azure Speech languages. xAI Grok STT integration for real-time transcription. Model: xai/grok-stt Languages: 25 languages, including Arabic, English, French, German, Hindi, Japanese, Korean, Portuguese, Spanish, and Vietnamese. AssemblyAI Universal-Streaming integration for real-time voice agent transcription. Model: assemblyai/universal-streaming Languages: English, Spanish, German, French, Portuguese, and Italian. Speechmatics real-time transcription integration with high accuracy and multilingual support including bilingual packs. Model: speechmatics/standard Languages: English, Spanish, plus bilingual/multilingual packs including Arabic–English, Mandarin–English, English–Malay, English–Tamil, Tagalog, and Spanish–English bilingual. Also supports Basque, Galician, Irish, Maltese, Mongolian, Swahili, Uyghur, and Welsh. Features: Supports interim results (partial transcripts) and graceful CloseStream shutdown. Soniox real-time transcription integration with automatic language detection. Model: soniox/stt-rt-v4 Languages: Automatic detection — no language hint required. Features: Supports interim results (partial transcripts), endpointing, and graceful CloseStream shutdown. Self-hosted NVIDIA Parakeet integration for multilingual transcription with automatic language detection. Model: nvidia/parakeet-v3 Languages: Automatic detection — no language hint required. Features: Final transcripts only — no interim/partial results. Ignores endpointing. Accepts linear16/linear32 (16 kHz) audio only.

How to Choose

Need the highest accuracy for English? → Deepgram nova-3 — best WER (word error rate) across all English variants. Building a voice agent that needs to know when the user stopped talking? → Deepgram flux — lowest latency with built-in end-of-turn detection. Need to transcribe files in 50+ languages? → Telnyx openai/whisper-large-v3-turbo via REST API. Need diarization (who said what)? → Deepgram nova-3 with model_config.diarize: true. Need broad accent/dialect support? → Azure azure/fast — strong coverage across regional accents. Need Grok STT for real-time calls? → xAI xai/grok-stt via WebSocket or Voice API. Need low-latency streaming for voice agents? → AssemblyAI assemblyai/universal-streaming via WebSocket or Voice API. Need high-accuracy multilingual with bilingual packs? → Speechmatics speechmatics/standard via WebSocket or Voice API. Need real-time transcription with automatic language detection? → Soniox soniox/stt-rt-v4 via WebSocket or Voice API. Need self-hosted multilingual transcription with automatic language detection? → Parakeet nvidia/parakeet-v3 via WebSocket or Voice API.

Specifying the Engine and Model

WebSocket — set via query parameters:
REST API — set via the model body parameter:

Migration

Source: https://developers.telnyx.com/docs/voice/stt/migration.md
Most migrations require changing 2–3 lines of code. Pick your current provider below.

WebSocket

The wire protocol is the same — send binary audio frames, receive JSON transcripts. What changes Response field mapping

REST

What changes Response field mapping

WebSocket

The wire protocol is the same — send binary audio frames, receive JSON transcripts. Config mapping Response field mapping

REST

What changes Response shape: identical. Telnyx returns the same {"text": "..."} shape — no parsing changes needed.

REST

The easiest migration — Telnyx REST is OpenAI SDK compatible. Change the API key and base URL, everything else stays the same.
Python
JavaScript
What changes Response shape: Telnyx returns the same {"text": "..."} shape — no parsing changes needed for the default json response format. Note on verbose_json: OpenAI’s Whisper API returns segments with timestamps when you set response_format=verbose_json. Telnyx’s Whisper models (openai/whisper-large-v3-turbo, openai/whisper-tiny) return text only — no segments. If you need timestamps, switch to model="deepgram/nova-3" which supports segment- and word-level timestamps via model_config.

WebSocket

Google uses gRPC with protobuf. Telnyx uses WebSocket with JSON — no protobuf compilation, no service account credentials.
See the Quickstart for full WebSocket code. Config mapping What you drop: protobuf definitions, gRPC client setup, service account credentials. Response field mapping

WebSocket

AWS Transcribe Streaming uses HTTP/2 with event streams via the amazon-transcribe-streaming-sdk. Telnyx uses a plain WebSocket — no AWS SDK, no SigV4 signing, no IAM credentials.
Send binary audio frames, receive JSON transcripts. No AWS SDK, no SigV4 signing, no IAM. See the Quickstart for full code. Config mapping Response field mapping

WebSocket

The Azure Speech SDK wraps a region-specific WebSocket with SDK abstractions and event handlers. Telnyx uses a plain WebSocket — no SDK install, no region routing.
Send binary audio frames, receive JSON transcripts. No SDK install, no region routing. See the Quickstart for full code. Config mapping Response field mapping What you drop: Azure Speech SDK install, region routing, event-handler boilerplate, subscription key management.

REST

What changes Response field mapping

Lifecycle

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming.md
Real-time speech-to-text over a persistent WebSocket connection. Send audio, receive transcripts.

Endpoint

Connection Lifecycle

1. Handshake

The connection starts as an HTTP GET with Upgrade: websocket. The server responds with 101 Switching Protocols, then the connection upgrades to WebSocket frames.
You can also connect directly to the WebSocket endpoint without an HTTP upgrade:
The same query parameters apply. Once connected, the message protocol is identical. All configuration is set at connect time via query parameters — engine, model, format, language, options. Cannot be changed mid-session. See Parameters for the full list. Invalid parameters return a JSON error and the connection closes.

2. Streaming

Once connected, audio and transcription flow concurrently — no request/response pairing. Client → Server Server → Client See Messages for the complete wire protocol reference.

3. Teardown

Send {"type": "CloseStream"} (Deepgram, Speechmatics, and Soniox) to flush remaining audio and close gracefully. The server finishes processing, sends any remaining transcripts, then closes the WebSocket.
For other engines, close the WebSocket connection directly. Dropping the connection without CloseStream works but may lose buffered audio on Deepgram, Speechmatics, and Soniox. See Examples for complete code samples.

Overview

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters.md
All parameters are passed as query string values on the WebSocket URL. Locked at connection time — cannot be changed mid-session. Deepgram, Telnyx, Google, Azure, xAI, AssemblyAI, Speechmatics, Soniox, Parakeet See Engines & Models See Audio Formats Hz. Required for raw encodings (linear16, mulaw, alaw). Ignored for container formats. Invalid value returns error 40005. Language code or auto-detect behavior varies by engine. See Language for details. "true" for partial transcripts. See Interim Results for details. Silence detection in ms for engines that support endpointing. "false" to disable. See Endpointing for details. PII redaction. Deepgram only. See Redaction for details. Comma-separated boost terms. Deepgram Nova-3/Flux. See Keyword Boosting for details. Legacy keyword boosting with intensifiers. Deepgram Nova. See Keyword Boosting for details. Azure Speech Services region. Flux only. Confidence threshold for EndOfTurn events. See End-of-Turn Detection for details. Flux only. Max silence before forcing EndOfTurn. See End-of-Turn Detection for details. Flux only. Speculative EagerEndOfTurn threshold. Disabled by default. See End-of-Turn Detection for details.

Example


Audio Formats

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/audio-formats.md
Set via the input_format query parameter. Audio is sent as binary WebSocket frames — chunked bytes, no base64, no JSON wrapping. Container formats (mp3, webm, etc.) are self-describing: the server demuxes the byte stream and extracts encoding/sample rate from headers. Raw formats have no metadata, so you must set sample_rate explicitly. Works for both real-time capture (microphone, MediaRecorder, telephony bridge) and file streaming (read a file in chunks, push through the socket).

Browser Capture

Output from MediaRecorder or similar browser APIs. Container headers carry sample rate.

Telephony

Codecs from voice networks. Raw frames, sample_rate required.
Invalid sample rate returns error 40005.

Raw PCM

Uncompressed audio from microphones, processing pipelines, or SDKs. sample_rate required.
Invalid sample rate returns error 40005.

Recorded File

Pre-recorded files read in chunks and streamed through the socket. Container headers carry sample rate.

Engine Compatibility

Unsupported format/engine combination returns error 40002. Unsupported Flux format returns error 40006. Deepgram has three model generations with different format support. Flux is the most restrictive of Deepgram’s models — it drops mp3, flac, webm_opus, amr_nb, amr_wb, g729, and speex compared to Nova. Universal formats (all engines and models): wav, linear16.

Engines & Models

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/engines-and-models.md
Set via transcription_engine and model query parameters.

Engines

Flux Model

Deepgram’s lowest-latency model with built-in end-of-turn detection. Designed for real-time voice agents. See Audio Formats for supported formats.

End-of-Turn Detection

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/end-of-turn.md
Deepgram Flux only. These parameters return a 400 error on non-Flux models. Flux uses a confidence-based system to decide when a speaker has finished their turn. Confidence threshold (0.50.9) for triggering an EndOfTurn event. Higher values require more certainty the speaker is done — fewer false positives but slightly more latency. Lower values respond faster but may cut speakers off mid-thought. Confidence threshold (0.30.9) for triggering an early EagerEndOfTurn event. Not set by default — setting it enables eager mode. When fired, your agent can start generating a response speculatively. If the speaker resumes, a TurnResumed event cancels it. Must be ≤ eot_threshold. Lower values = earlier triggers, more false starts. Typical range: 0.30.5 for ~150–250 ms latency savings at the cost of ~50–70% more LLM calls. Maximum silence in ms (50010000) before forcing EndOfTurn regardless of confidence. Resets when speech resumes. Increase for speakers who pause frequently; decrease for rapid-fire Q&A.

Event Flow

Without eager mode (eot_threshold only):
With eager mode (eager_eot_threshold set):

Configuration Profiles

Default — balanced for general use:
Low-latency — fast response, more false starts:
High-reliability — fewer interruptions, more latency:

Language

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/language.md
BCP-47 language code. Default: en-US.

Auto-Detection

Pass multi to enable automatic language detection. The aliases auto and auto_detect are silently mapped to multi.

Engine Support

Supported Languages

For most engines, Telnyx passes the language code directly without validation. The supported set depends on which engine you use. Speechmatics is the exception — Telnyx accepts shorthand codes for bilingual/multilingual packs and maps them to the provider’s language + domain configuration internally.

Common Codes


Interim Results

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/interim-results.md
Deepgram, Speechmatics, and Soniox. Other engines ignore this parameter. Controls whether the server sends partial (non-final) transcripts as speech is processed.
Default: false. Pass "true" as a string. interim_results=false (default) — Server sends only final transcripts. Each message has is_final: true. Lower message volume, higher latency per result.
interim_results=true — Server sends evolving partial transcripts as audio is processed, followed by a final. Partials have is_final: false and are replaced by the next message.
Partial transcripts have confidence: 0.0 — confidence is only meaningful on final results.

Endpointing

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/endpointing.md
Deepgram, xAI, Google, Speechmatics, and Soniox. Other engines ignore this parameter. Controls how long the engine waits after silence before finalizing an utterance. Soniox has a different valid range. When transcription_engine=Soniox, this parameter maps to max_endpoint_delay_ms and must be between 500 and 3000 ms. Values outside that range are rejected. The default (100 ms) and the low-value examples below apply to Deepgram, xAI, Google, and Speechmatics only.
Default: 100 ms (not applicable to Soniox — Soniox endpointing is disabled unless a value in the 500–3000 ms range is provided).

Values

Trade-offs

Low values (50–100 ms) — Fast response. Utterances may split mid-sentence on short pauses. (Deepgram, xAI, Google, Speechmatics only — below Soniox minimum.) High values (300–1000 ms) — More complete sentences. Higher latency before finalization. Soniox range (500–3000 ms) — Minimum 500 ms. Use 500–800 ms for responsive turn detection, 1000–3000 ms for longer utterances with natural pauses. Disabled ("false") — No automatic splits. Use Finalize control messages to manually trigger boundaries, or rely on CloseStream for a single final transcript.

Interaction With Utterance End

When endpointing triggers, Deepgram sends the final transcript followed by an utterance end event (if utterance_end_ms is configured server-side — currently 1000 ms).
The utterance end marker signals “this speaker turn is done.” See Messages for details.

Keyword Boosting

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/keyword-boosting.md
Deepgram only. Other engines ignore these parameters. Two parameters control keyword boosting. They target different Deepgram model generations.

keyterm — Nova-3 and Flux

Comma-separated list of terms to boost. Simple — no intensifiers.
Deepgram Nova-3 and Flux only. Ignored on older models.

keywords — Nova (Legacy)

Terms with optional intensity scores. Format: keyword:intensifier.
Deepgram Nova only. Not supported on Flux (silently ignored).

Which To Use

Examples

Boost multiple terms on Nova-3:
Boost with intensifiers on legacy Nova:

Redaction

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/redaction.md
Deepgram only. Other engines ignore this parameter. Replaces sensitive data in transcripts with placeholder text.

Values

Multiple values can be passed as comma-separated:

Output

Redacted content is replaced in the transcript text. The exact replacement format depends on the Deepgram model.
Redaction applies to final and interim results. There is no way to get the un-redacted version once redaction is enabled for a session.

Messages

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/responses.md
The WebSocket carries two frame types: binary frames (audio) from client to server, and JSON text frames in both directions.

Client → Server

Audio Data

Binary WebSocket frames containing raw audio bytes. No base64, no JSON wrapping. Recommended chunk size: 2048–8192 bytes. Smaller chunks reduce latency; larger chunks reduce round trips.

Control Messages

JSON text frames with a type field.
Finalize
CloseStream
KeepAlive
Unknown text frames are silently ignored.

Server → Client

All server messages are JSON text frames.

Transcription Result

Emitted for each recognized speech segment (partial or final).

Utterance End

Emitted on speaker pause (Deepgram). Empty transcript, is_final: true.

Error

Emitted on validation or connection errors. Connection closes shortly after.

Message Flow

interim_results=false (default) — server sends only final transcripts:
interim_results=true — server sends partials, then final:
Partials are best-effort and may revise. Only is_final: true results are stable.

Errors

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/errors.md
If you send an invalid parameter (unsupported engine, format, or format/engine combination), the server responds with a JSON error and closes the connection.

Error Response Format

Error Codes


Examples

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/examples.md
Stream a WAV file and print transcripts.
Python
JavaScript

Production Patterns

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/production-patterns.md
Use these patterns when running the standalone WebSocket STT endpoint in production.

Connection Recovery

Treat the WebSocket session as disposable. Reconnect on network failure, server close, idle timeout, and process restart. Set all query parameters on every reconnect. STT configuration cannot be changed mid-session.

Backoff

Use bounded exponential backoff with jitter. Add random jitter of 0-500 ms per attempt. Reset the attempt counter after a stable connection. Do not retry immediately on authentication or validation errors. Fix the API key, query parameters, engine, model, or format first.

Partials

Enable interim_results=true when the application needs live captions or low-latency UI updates. Store final transcript segments separately from the current partial. This prevents duplicate text when a final result arrives after one or more interim results.

Audio Buffering

Buffer audio at the producer boundary, not inside the WebSocket send loop. Avoid unbounded queues. A slow or disconnected socket should not grow memory usage indefinitely. For live audio, prefer dropping stale buffered audio over sending it late. Late audio increases transcript delay and can make captions appear out of sync.

Keepalive

For Deepgram sessions, send {"type": "KeepAlive"} during long silence periods. Keep sending audio as binary frames when audio is available. For other engines, use the WebSocket client’s ping/pong support when available and reconnect on missed heartbeats.

Monitoring

Track connection, latency, transcript, and buffer metrics. Log the selected transcription_engine, model, input_format, sample_rate, and interim_results value with each session. Redact API keys and user audio.

Shutdown

Use graceful shutdown for planned stops.
  1. Drain the audio queue.
  2. Send {"type": "CloseStream"}.
  3. Wait for final transcript messages.
  4. Close the WebSocket.
Set a shutdown timeout. If final messages do not arrive before the timeout, close the socket and mark the transcript as incomplete.

Pricing

Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/pricing.md
Pricing for WebSocket STT varies by engine and model. Contact sales or check the pricing page for current rates.

Overview

Source: https://developers.telnyx.com/docs/voice/stt/rest-api.md
POST /v2/ai/audio/transcriptions Synchronous file transcription. Upload audio or pass a URL, get text back.

Feature Support

If you’re coming from alternative providers:

Quick Start

Python
JavaScript

Overview

Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters.md
All parameters are sent as multipart/form-data. Model to use for transcription. See Models for details. Values: openai/whisper-large-v3-turbo (default), openai/whisper-tiny, deepgram/nova-3 Audio file to transcribe. Mutually exclusive with file_url. See Audio Formats for supported formats, size limits, and per-model restrictions. Publicly accessible URL to an audio file. Mutually exclusive with file. See Audio Formats for details on how file and file_url differ. Language hint. Behavior varies by model — see Language. Output shape. See Response Format. Values: json (default), verbose_json Timestamp detail level. Only valid with response_format=verbose_json — returns 400 otherwise. Values: segment Deepgram-specific options. Only valid with deepgram/nova-3 — returns 400 for other models. See Model Config.

Models

Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters/models.md
Your choice of model determines which audio formats are accepted, what language values are valid, and what response fields are available.

openai/whisper-large-v3-turbo

Default model. Multilingual. Auto-detected if language omitted. See Whisper docs for the full language list. Returns text only — no timestamps regardless of response_format.

openai/whisper-tiny

Lightweight, lowest resource usage. Multilingual (50+ languages, auto-detected). Returns text only — no timestamps.

deepgram/nova-3

Highest accuracy for English. Advanced features (diarization, word timestamps, smart formatting, numerals, punctuation) available via model_config. Defaults language to en if omitted. Can also set language inside model_config — top-level field takes precedence. See Deepgram language docs for details.

Audio Formats

Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters/audio-formats.md
Applies to both file (multipart upload) and file_url (URL download).

Common

  • Max size: 100 MB
  • Processing: All audio is decoded, resampled to 16kHz, and mixed to mono via ffmpeg before transcription. Container format doesn’t matter as long as ffmpeg can decode it — the validated extension list is the actual restriction.

Supported Formats

file vs file_url

One of file or file_url is required. Sending both returns 400.

Model Config

Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters/model-config.md
Deepgram only. Returns 400 if used with other models. Pass-through to Deepgram’s pre-recorded API query parameters. Every key-value pair in model_config is forwarded directly — Telnyx does not validate individual options.

Commonly Used Options

Example

model_config can be sent as a JSON string in the multipart form field. The server parses it before forwarding.

Unvalidated Pass-Through

Any Deepgram query parameter can be passed. If Deepgram adds new options, they work immediately without a Telnyx API update. Conversely, invalid keys are forwarded and may cause Deepgram to return an error. Refer to Deepgram’s API reference for the full list of supported parameters.

Response Format

Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters/response.md
Controlled by the response_format parameter.

json (Default)

Text only.

verbose_json

Adds duration (seconds) and timestamped segmentsonly when using deepgram/nova-3. The Whisper models (openai/whisper-large-v3-turbo, openai/whisper-tiny) return text only regardless of response_format. See the Timestamp Availability table below. Example response with model=deepgram/nova-3:
Set timestamp_granularities[]=segment alongside response_format=verbose_json. Using timestamp_granularities without verbose_json returns 400.

Segment Fields

Timestamp Availability by Model

Streaming Response (Undocumented)

Sending Accept: application/stream+json returns newline-delimited JSON chunks as segments are transcribed. Each line:
This is used internally but not in the public OAS spec.

Pricing

Source: https://developers.telnyx.com/docs/voice/stt/rest-api/pricing.md
Pricing for REST API STT varies by engine and model. Contact sales or check the pricing page for current rates.

In-Call Transcription

Source: https://developers.telnyx.com/docs/voice/stt/in-call-transcription.md
In-call transcription enables real-time STT on live voice calls. The audio codec is managed by the Telnyx platform — no format configuration needed. Two integration paths: Engine selection (Telnyx, Google, Deepgram, Azure, xAI, AssemblyAI, Speechmatics, Soniox, Parakeet) is specified as a parameter on the transcription command. Same engines as WebSocket streaming.

TTS

Overview

Source: https://developers.telnyx.com/docs/voice/tts/overview.md

1. Choose your interface

Real-time streaming. Send text, receive audio chunks as they’re synthesized. HTTP POST. Get audio back as binary, base64, or async URL. OpenAI SDK compatible. TTS during live calls via Call Control speak or TeXML “.

2. Choose a pre-built voice

Natural, NaturalHD, Ultra, Kokoro, Qwen3TTS, xAI Grok. AWS Polly, Azure, ElevenLabs, Minimax, MurfAI, Rime, Resemble, Inworld, Fish Audio.

3. Or create your own

Clone and design custom voices. Available on select providers: Qwen3TTS, Minimax, ElevenLabs, Resemble.

Lifecycle

Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming.md
Real-time text-to-speech over a persistent WebSocket connection. Send text, receive audio.

Endpoint

Connection Lifecycle

1. Handshake

There are two ways to establish a connection:

Direct WebSocket connection

You can connect directly to the WebSocket endpoint by passing all configuration as query parameters in the wss:// URL:
Most WebSocket clients and libraries support this natively — simply open a WebSocket connection to the URL and begin the message flow. No separate HTTP request is needed.

HTTP upgrade

Alternatively, initiate the connection as an HTTP GET request that upgrades to a WebSocket via the standard 101 Switching Protocols handshake. This is what happens under the hood when a WebSocket client connects, and may be relevant if you need fine-grained control over the upgrade (e.g., setting custom headers in environments where the WebSocket library doesn’t expose them directly).

Initialization frame

Regardless of how the connection is established, send an initialization frame before any text:
The initialization frame may include voice_settings to configure provider-specific parameters:
All configuration — query parameters and voice settings — is locked before synthesis begins. See Configuration for both surfaces and the full parameter reference.

2. Streaming

Once initialized, text and audio flow concurrently — no request/response pairing. Text is buffered and synthesized at sentence boundaries. Client → Server Server → Client See Messages for the complete wire protocol reference.
Text buffering: Text accumulates until the server detects a sentence boundary (period, question mark, exclamation). Short fragments without punctuation wait for more text. Send "flush": true to force synthesis of buffered partials. Text preprocessing: Markdown formatting is automatically stripped before synthesis (headers, bold, italics, code blocks, links, lists, emoji). Useful when synthesizing LLM output. Pronunciation dictionary replacements are applied if pronunciation_dict_id is set. Streamed vs. concatenated delivery: Most providers (Telnyx Natural/NaturalHD/Qwen3TTS, Rime, Minimax, Resemble, Inworld) stream audio in separate frames where text is null. AWS Polly and Azure return audio in the text-bearing chunk instead. See Messages for details.

3. Teardown

Send {"text": ""} (empty string) to flush remaining buffered text and close gracefully. The server finishes synthesis, sends any remaining audio and a final frame, then closes the WebSocket.
Dropping the connection without the empty-text frame works but may lose buffered text. The connection also closes on server error or inactivity timeout. See Examples for complete code samples.

Configuration

Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming/configuration.md

Two Configuration Surfaces

Both are one-shot. After the init frame, no configuration can change for the session. To change settings, open a new connection.

Query Parameters

Set on the URL at connect time. Immutable for the session.

Voice Selection

The voice_id segment (third part of the voice string) refers to different things depending on the provider and model: The Voices API (GET /v2/ai/tts/voices) returns all voices available to your account — pre-built and cloned — with each voice’s compound id ready to use as the voice parameter.

Connection Options

| disable_cache | boolean | false | Bypass the audio cache and always synthesize fresh. |

Example

Voice Settings

Provider-specific tuning (speed, pitch, format, emotion, etc.) is not set via query parameters. It is passed once in the voice_settings object on the initialization frame:
Voice settings are applied when the synthesis worker starts and cannot be changed mid-session. There are no common voice_settings fields. Every field is provider-specific — the available fields, defaults, and accepted values are completely different per provider. Unrecognized fields are silently ignored. See your selected provider’s page under Providers for the exact fields.

Messages

Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming/messages.md

Client → Server

All client messages are JSON text frames.

Text Frame

Text to synthesize. " " (single space) for handshake. "" (empty string) for end-of-sequence. Provider-specific voice configuration. Only used in the handshake frame ({"text": " "}). See Voice Settings. When true, immediately synthesizes all buffered text without waiting for a sentence boundary. Default: false. When true, stops the current synthesis worker and starts a new one. The original handshake is replayed automatically. Use for barge-in/interruption.

Message Sequence

1. Handshake (required first message):
With optional voice settings:
2. Text (one or more):
3. Flush (optional — force synthesis of buffered partial sentences):
4. Interrupt (optional — restart synthesis):
5. End of sequence:

Server → Client

All server messages are JSON text frames.

Audio Chunk

Returned when synthesis produces audio for a complete sentence.
Base64-encoded audio data. null when the provider uses streamed delivery — audio arrives in separate streamed chunk frames instead. See note below. The text segment this audio corresponds to. null for streamed audio chunks. false for audio chunks. true if audio was served from cache. Time in milliseconds from speech request to first audio frame. Only present on the first chunk of each synthesis.

Streamed Audio Chunk

For providers that stream audio incrementally (Telnyx Natural, NaturalHD, Qwen3TTS, Rime, Minimax, Resemble, Inworld), audio arrives in separate frames:
These contain raw audio data (text is always null). The concatenated audio chunk for these providers has audio: null — only the streamed chunks carry audio bytes. For AWS Polly and Azure, audio is returned in the audio field of the regular audio chunk frame. For all other providers, ignore the audio field on the text-bearing chunk and collect audio from the streamed frames.

Final Frame

Signals that synthesis is complete for the current text input:
The connection remains open after a final frame — send more text or close.

Error Frame

The connection closes shortly after an error frame.

Errors

Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming/errors.md

HTTP Errors (Handshake)

These occur during the WebSocket upgrade request, before the connection is established.

WebSocket Errors (Runtime)

After the connection is established, errors arrive as JSON frames:
The connection closes shortly after an error frame.

Error Messages

Troubleshooting


Examples

Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming/examples.md

Basic Streaming

Python
JavaScript

Conversational (Barge-In)

Send multiple text segments and interrupt mid-synthesis:

LLM Token Streaming

Stream tokens from an LLM directly to TTS. The server buffers text and synthesizes at sentence boundaries:
Markdown in LLM output is automatically stripped before synthesis — headers, bold, italics, code blocks, and links are converted to plain text.

Overview

Source: https://developers.telnyx.com/docs/voice/tts/rest-api.md

How It Works

You send text in, audio streams back over the same HTTP connection. No polling, no callbacks. The response uses HTTP chunked transfer encoding — audio chunks arrive as they’re synthesized. Your client can begin playback immediately without waiting for the full file. The connection stays open until synthesis completes or 30 seconds pass with no new chunks. This makes REST suitable for real-time playback, not just batch file generation. For multi-turn conversational use cases where you’re continuously feeding text, use WebSocket Streaming instead.

Text Preprocessing

Before synthesis, text passes through two stages:
  1. Markdown stripping — headers, bold, italics, code blocks, links, lists, emoji are converted to plain text.
  2. Pronunciation dictionary — if pronunciation_dict_id is set, custom word replacements are applied.

API Reference

The full OpenAPI spec for these endpoints is available in the auto-generated API Reference. Note: the OAS is currently being cleaned up — some fields and provider-specific schemas may be incomplete.

Request

Source: https://developers.telnyx.com/docs/voice/tts/rest-api/request.md

Endpoint

Example

Request Body


Response

Source: https://developers.telnyx.com/docs/voice/tts/rest-api/response.md
The output_type request field controls what comes back.

Streaming Audio (default)

With output_type: "binary_output" (or omitted), the response is raw audio over HTTP chunked transfer encoding:
Start reading the body immediately — don’t buffer the full response.

Base64

With output_type: "base64_output", the full audio is returned as a JSON payload after synthesis completes:
No streaming — the entire file must synthesize before the response is sent.

Async (audio_id)

With output_type: "audio_id", synthesis runs in the background. You get a URL back immediately:
Retrieve the audio later with GET /v2/text-to-speech/speech/:audio_id. If the audio is still synthesizing, the GET response itself streams chunks as they become available.

Examples

Source: https://developers.telnyx.com/docs/voice/tts/rest-api/examples.md

OpenAI SDK Compatibility

The REST endpoint is a drop-in replacement for the OpenAI Audio API:

API Reference

Source: https://developers.telnyx.com/docs/voice/tts/rest-api/api-reference.md
See the full API reference: Generate Speech from Text.

Overview

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx.md

Selecting Telnyx

Telnyx is the default provider. If you don’t specify a provider, you get Telnyx. WebSocket:
REST:

Models

Ultra is REST-only, not available over public WebSocket. Grok is available for Voice AI Assistants and direct REST TTS calls.
Browse all available voices via the Voices API or the Voice Design.

Natural

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/natural.md
Voice format: Telnyx.Natural. Pre-built English voices backed by Rime Mist.

Voice Samples


WebSocket

Query Parameters

Voice Settings

Send in the init frame ({"text": " "}):

REST API

Fields

Response

Default (binary_output): chunked audio bytes with Content-Type: audio/mpeg (or audio/wav, audio/pcm). With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

NaturalHD

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/naturalhd.md
Voice format: Telnyx.NaturalHD. Pre-built voices backed by Rime Arcana. 9 languages: en, fr, de, es, ar, hi, ja, he, pt.

Voice Samples


WebSocket

Query Parameters

Voice Settings

Send in the init frame ({"text": " "}):

REST API

Fields

Response

Default (binary_output): chunked audio bytes with Content-Type: audio/mpeg (or audio/wav, audio/pcm). With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

KokoroTTS

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/kokoro.md
Voice format: Telnyx.KokoroTTS. Lightweight, lowest-latency model. 5 languages: en, es, fr, it, pt.

Voice Samples


WebSocket

Query Parameters

Voice Settings

None. All synthesis parameters are fixed. The init frame only needs {"text": " "}.

REST API

Fields

No model-specific fields. Audio format is always MP3.

Response

Default (binary_output): chunked audio bytes with Content-Type: audio/mpeg. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

Qwen3TTS

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/qwen3.md
Voice format: Telnyx.Qwen3TTS. Voice cloning model. 11 languages: en, zh, fr, de, it, ja, ko, pt, ru, es, ar. The voice_id is the name of a clone you created in the Voice Design. Clones are scoped to your organization.

Voice Samples


WebSocket

Query Parameters

Voice Settings

Send in the init frame ({"text": " "}):

REST API

Fields

Response

Default (binary_output): chunked PCM audio bytes. Always 24kHz signed 16-bit LE mono. With output_type: "base64_output": JSON with base64-encoded PCM. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

Ultra

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/ultra.md
Voice format: Telnyx.Ultra. Sub-100ms latency. 36 languages. REST only — Ultra is not available over public WebSocket.

Voice Samples


SSML Emotions

Ultra supports inline SSML emotion tags. Place the tag before the text:
Primary emotions: angry, excited, content, sad, scared. Additional: happy, enthusiastic, curious, calm, grateful, affectionate, sarcastic, surprised, confident, hesitant, apologetic, determined, frustrated, disappointed. Omitting the tag = neutral delivery. Use sparingly — Ultra interprets emotional subtext from the text itself.

Nonverbal Cues

Insert [laughter] inline for natural laughing:

Language Support

Set language_boost to improve pronunciation for the target language: Arabic, Bengali, Bulgarian, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Gujarati, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Marathi, Māori, Norwegian, Polish, Portuguese, Punjabi, Romanian, Russian, Slovak, Spanish, Swedish, Tamil, Telugu, Thai, Turkish, Ukrainian, Vietnamese.

REST API

Fields

Response

Default (binary_output): chunked audio bytes with Content-Type: audio/mpeg. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

See also

xAI Grok is the second TTS provider supporting Expressive Mode. For Grok voice options, see Grok Voices. Note: Grok voices have higher latency than Ultra.

Grok

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/grok.md
xAI Grok voices are expressive text-to-speech voices for Voice AI Assistants. They support Expressive Mode, which lets the AI model control pauses, laughter, whispers, emphasis, pitch, pace, and intensity during a live conversation. Higher latency: Grok voices have higher latency than Ultra. For latency-sensitive applications that need sub-100ms time to first byte, use Ultra.

What makes Grok voices different

Voice format

For AI Assistants, Grok voices use the format:
Examples:

Voices

Voice Samples

Expressive Mode for AI Assistants

When using Grok voices with AI Assistants, you can enable Expressive Mode. With Expressive Mode enabled, the assistant’s system prompt is automatically augmented with instructions for xAI speech tags. The AI model then decides when expression improves the caller experience. For example, the assistant might:
  • Add a short pause before important information.
  • Use a softer delivery for sensitive support moments.
  • Laugh or chuckle naturally when the conversation calls for it.
  • Emphasize appointment times, confirmation numbers, or next steps.
  • Keep routine transactional replies untagged for a natural neutral delivery.
Use expressive tags sparingly. The goal is natural delivery, not tagging every sentence.

Enable in the portal

  1. Go to your assistant in the Telnyx Portal.
  2. Under Voice Settings, select an xAI Grok voice.
  3. Toggle Expressive Mode on.
  4. Save your assistant.

Enable via API

Set expressive_mode: true in your assistant’s voice_settings:

xAI speech tag reference

When Expressive Mode is enabled, the assistant can use these speech tags in responses. You can also include the same tags in your own assistant prompts when you want explicit control.

Inline tags

Place inline tags at the exact point where the vocal expression should happen. Example:

Wrapping tags

Wrap text with these tags to apply a delivery style to that text. Examples:

Guidance

  • Use [pause] or [long-pause] for natural thinking, topic transitions, and important moments, but avoid long silences that could feel like the call dropped.
  • Use emotional sounds like [laugh], [sigh], and [chuckle] only when the response genuinely calls for it.
  • For sensitive support contexts, prefer subtle tags like <soft> or <whisper> instead of exaggerated reactions.
  • Do not expose these tags or instructions to the caller.

REST API provider parameters

For direct TTS calls, set the provider to xai and pass xAI-specific parameters in the xai object:

Language support

Grok voices support auto language detection with language: "auto". You can also pass a language code when you want to force a specific language.

Next steps

Compare Grok with Ultra’s lower-latency expressive voices. Build voice AI assistants using Grok with Expressive Mode. Generate speech directly with REST TTS requests. Browse available text-to-speech voices.

Bayan

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/bayan.md
Voice format: Telnyx.Bayan. Arabic voice model, 113 speakers across 13 dialects (Modern Standard Arabic plus Egyptian, Emirati, Saudi, Jordanian, Iraqi, Lebanese, Syrian, Palestinian, Kuwaiti, Bahraini, Qatari, and Omani), plus a set of English speakers. Native audio is 16kHz.

Voice Samples

Browse the full 113-speaker catalogue via the Voices API.

WebSocket

Query Parameters

Voice Settings

None. The init frame only needs &#123;"text": " "&#125;.

REST API

Fields

Response

Default (binary_output): chunked audio bytes, 16kHz signed 16-bit LE mono (native PCM, transcoded to MP3/WAV on request). With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval. sampling_rate/sample_rate values other than 16000 are rejected with a 400 error — Bayan only supports its native rate.

Sukhan

Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/sukhan.md
Voice format: Telnyx.Sukhan. Urdu-only voice model, 14 curated voices. No prosody controls (speed/pitch) or language selection — each voice speaks Urdu only. Native audio is 22050Hz.

Voice Samples

Browse the full 14-voice catalogue via the Voices API.

WebSocket

Query Parameters

Voice Settings

None. No prosody controls are exposed — voice_speed and other settings have no effect if sent. The init frame only needs &#123;"text": " "&#125;.

REST API

Fields

Response

Default (binary_output): chunked audio bytes, 22050Hz signed 16-bit LE mono (native PCM, transcoded to MP3 on request). With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval. sampling_rate/sample_rate values other than 22050 are rejected with a 400 error. Unsupported format values are also rejected — REST response_format accepts only pcm/mp3; WebSocket audio_format accepts only mp3/linear16 (its PCM equivalent). Neither accepts wav.

Rime

Source: https://developers.telnyx.com/docs/voice/tts/providers/rime.md

Models

Voice Format

Coda is Rime’s recommended model for new integrations. It surpasses ArcanaV3 in naturalness, prosody, and artifact-free output.

Voice Samples


WebSocket

Query Parameters

Voice Settings

REST API

Fields

Response

Default (binary_output): chunked audio bytes. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

Minimax

Source: https://developers.telnyx.com/docs/voice/tts/providers/minimax.md
Voice format: minimax.. voice_id can be a system voice (pre-built) or a cloned voice from the Voice Design (organization-scoped).

Voice Samples


WebSocket

Query Parameters

Voice Settings

REST API

Fields

Response

Default (binary_output): chunked audio bytes. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

Resemble

Source: https://developers.telnyx.com/docs/voice/tts/providers/resemble.md
Voice format: resemble.Turbo. Default model: Turbo. voice_id is a voice from your own Resemble account.

Voice Samples


WebSocket

Query Parameters

Voice Settings

REST API

Fields

Response

Default (binary_output): chunked audio bytes. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

Inworld

Source: https://developers.telnyx.com/docs/voice/tts/providers/inworld.md
Voice format: inworld.. Models:
  • inworld-tts-1.5-mini (alias Mini) — faster, lower latency.
  • inworld-tts-1.5-max (alias Max) — higher quality.
  • inworld-tts-2 (alias TTS2) — latest generation; supports the delivery_mode parameter.
Defaults to inworld-tts-1.5-mini if model omitted.

Voice Samples


WebSocket

Query Parameters

Voice Settings

REST API

Fields

Response

Default (binary_output): chunked audio bytes. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

xAI

Source: https://developers.telnyx.com/docs/voice/tts/providers/xai.md
Voice format: xAI. xAI Grok voices are expressive, multilingual text-to-speech voices. They support inline speech tags for pauses, vocal sounds, emphasis, pitch, pace, and intensity. xAI Grok voices are higher-latency than Telnyx Ultra. For latency-sensitive applications that need sub-100ms time to first byte, use Ultra.

Voices

Voice Samples


WebSocket

xAI Grok voices are not available on the public TTS WebSocket API. Use the REST API for direct text-to-speech generation, or use xAI Grok voices with AI Assistants.

REST API

Fields

Response

Default (binary_output): chunked audio bytes. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

Expressive speech tags

Use speech tags inline in text when you want more expressive delivery.
Use expressive tags sparingly. The goal is natural delivery, not tagging every sentence.

AI Assistants

For AI Assistants, choose an xAI Grok voice such as xAI.eve and enable Expressive Mode to let the assistant decide when speech tags improve the caller experience. Build voice AI assistants using xAI Grok voices with Expressive Mode. Generate speech directly with REST TTS requests.

Fish Audio

Source: https://developers.telnyx.com/docs/voice/tts/providers/fishaudio.md
Voice format: FishAudio.. Fish Audio is a Telnyx-hosted external TTS provider. Telnyx exposes a hand-vetted shortlist of Fish Audio voices from their public Voice Library. Fish Audio is cross-lingual — any voice can speak any language from the input text. The language field in the voice listing is the voice’s native accent only, not a synthesis constraint.

Models

All curated voices are available under all three models.

Curated Voices

Only curated catalog voices are accepted for synthesis. Arbitrary Fish Voice-Library reference_ids cannot be used under the Telnyx API key.

Emotion Tags

Fish Audio S2 models (s2.1-pro, s2-pro) support inline emotion, tone, and audio-effect markers placed directly in the input text. S1 uses parentheses — (happy) — instead of brackets. The markers are processed by the upstream Fish Audio API and do not add latency or count toward token limits.
The same syntax works on both the REST endpoint and the WebSocket text field. Markers can be combined for layered effects — [excited][laughing] We won! Ha ha! — and accept intensity modifiers like [slightly sad], [very excited], [extremely angry]. For the full list of supported emotions, tone markers, audio effects, and S1 syntax, see Fish Audio’s Emotion Control guide.

WebSocket

Query Parameters

Voice Settings

Fish Audio does not expose speed or pitch controls. voice_settings are limited to format and sample rate overrides. Emotion and tone are controlled inline in the text via bracket markers — see Emotion Tags below.

REST API

Fields

Response

Default (binary_output): chunked audio bytes. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

Audio Formats & Sample Rates

Each format has a fixed set of accepted sample rates: On WebSocket, the audio_format query parameter and voice_settings.format are independent parameters with different vocabularies. The audio_format query param accepts mp3, wav, and linear16 (which maps to pcm). The voice_settings.format field accepts mp3, wav, pcm, and opus directly. If both are set, voice_settings.format takes precedence.

In-Call Playback

Fish Audio is available for in-call TTS via Call Control speak and TeXML “. For telephony playback, the gateway automatically delivers MP3 at 44.1 kHz to ensure correct resampling by the call-control audio pipeline. See In-Call Playback for details.

Provider Docs


AWS Polly

Source: https://developers.telnyx.com/docs/voice/tts/providers/aws.md
Voice format: aws.Polly.. Example: aws.Polly.Generative.Lucia The engine can also be parsed from a hyphenated suffix on the voice ID — e.g., Lucia-longform resolves to engine long-form.

Voice Samples


WebSocket

Query Parameters

Voice Settings

REST API

Fields

Response

Default (binary_output): chunked audio bytes. Format depends on output_format. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

SSML Tags

Source: https://developers.telnyx.com/docs/voice/tts/providers/aws/ssml-tags.md
In this tutorial, you’ll learn about SSML tags that can help customize your audio response in your text-to-speech application. This tutorial assumes you’ve already set up your developer account and environment and you know how to send commands and receive webhooks using the Telnyx Voice API.

What are SSML tags?

Speech Synthesis Markup Language (SSML) is an XML-based markup language that is used to generate synthetic speech for appliations. SSML tags are used to change the tone of speech in the application by adjusting pitching, volume, duration of speech, and more.

SSML tag examples

Adding a pause

SSML Tag: There are 2 ways for defining the length of the pause by using the following attributes:
  1. Time: Defines the number of s or ms
  2. Strength: Chooses the strength using the following values:
  • None: no pause
  • Pause: the same duration as after a period
  • x-weak: the same as none.
  • weak: sets a pause of the same duration as the pause after a comma
  • medium: has the same strength as weak
  • strong: sets a pause of the same duration as the pause after a sentence
  • x-strong: sets a pause of the same duration as the pause after a paragraph.
Example:

Emphasizing words

SSML Tag:
The emphasising affects the speed and loudness of reading words and can be defined by using a ‘level’ attribute with one of the following values:
  1. Strong - increases the volume and slows the speaking rate
  2. Moderate - increases the volume and slows the speaking rate, but less than Strong
  3. Reduced - decreases the volume and speeds up the speaking rate
Example:

Set a different language

SSML Tag:
The xml:lang tag defines the language for a specific word or sentence. __Example: __

Adding a pause between paragraphs

SSML Tag:
This tag adds a pause between paragraphs that is longer than a regular pause at a comma or at the end of the sentence. Example:

Using phonetic pronunciation

SSML Tag:
The phonetic pronunciation requires 2 attributes:
  1. Alphabet, with the following options:
  • ipa, meaning the International Phonetic Alphabet (IPA) will be used
  • x-sampa, which indicates that the Extended Speech Assessment Methods Phonetic Alphabet (X-SAMPA) will be used.
  1. ph, specifies how the text should be pronounced.
Example:

Controlling volume, speaking rate, and pitch

SSML Tag::
The following attributes can be used with the Prosody tag:
  1. Volume:
  • default: resets the volume to default value
  • silent, x-soft, soft, medium, loud, x-loud: sets the volume to predefined value
  • +ndB, -ndB: changes the volume relative to the current level
  1. Rate:
  • x-slow, slow, medium, fast,x-fast: sets the pitch to a predefined value
  • n%: a percentage change in speaking pace.
Exmaple:

Adding a pause between sentences

SSML Tag:
This tag adds a pause between lines with the same effect as (.) Example:

Controlling how special words are spoken

SSML Tags:
The say-as tag uses one attribute,‘interpret-as’, which uses a number of possible available values:
  • characters or spell-out
  • cardinal or number
  • digits
  • fraction
  • unit
  • date
  • time
  • address
  • telephone.
Example:

Pronouncing acronyms and abbreviations

SSML Tag:
This tag should be used with the alias attribute to substitute a different word for selected text such as an acronym or abbreviation. Example:

Azure

Source: https://developers.telnyx.com/docs/voice/tts/providers/azure.md
Voice format: azure. Example: azure.en-US-AvaMultilingualNeural No model ID segment — Azure voices are flat identifiers. Default voice: en-US-AvaMultilingualNeural.

Voice Samples


WebSocket

Query Parameters

Voice Settings

REST API

Fields

Response

Default (binary_output): chunked audio bytes. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

ElevenLabs

Source: https://developers.telnyx.com/docs/voice/tts/providers/elevenlabs.md
ElevenLabs requires your own API key configured in your Telnyx account. Telnyx relays requests to the ElevenLabs API — voice settings are passed through directly. Voice format: elevenlabs.. Example: elevenlabs.v3.Adam voice_id is a voice from your own ElevenLabs account — pre-built, cloned, or designed. Preview voices at elevenlabs.io/voice-library.

WebSocket

Query Parameters

Voice Settings

Relayed directly to the ElevenLabs API. Any field ElevenLabs accepts can be passed here.

REST API

Fields

Response

Default (binary_output): chunked audio bytes. Format determined by ElevenLabs. With output_type: "base64_output": JSON with base64-encoded audio. With output_type: "audio_id": JSON with an audio_url for deferred retrieval.

Pronunciation Dictionaries

Source: https://developers.telnyx.com/docs/voice/tts/pronunciation-dictionaries.md
Pronunciation dictionaries let you control how specific words and phrases are spoken during text-to-speech synthesis. Dictionaries are applied automatically before speech generation — no changes to your text input required.

Item Types

Each dictionary contains up to 100 items. Two types are supported:

Alias (text replacement)

Replaces matched text with alternative text before synthesis:

Phoneme (IPA notation)

Specifies exact pronunciation using the International Phonetic Alphabet:

Using a Dictionary

Pass the dictionary ID when synthesizing speech: REST API:
WebSocket: pass pronunciation_dict_id as a query parameter on the connection URL:

Managing Dictionaries

Create a dictionary

You can also upload a PLS/XML or plain text file via multipart/form-data instead of providing items as JSON. Plain text format:

List dictionaries

Get a dictionary

Update a dictionary

Updates use optimistic locking — if the dictionary was modified concurrently, the request returns 409 Conflict. Re-fetch and retry.

Delete a dictionary

Limits

File Upload Formats

When creating a dictionary via file upload, two formats are supported: PLS/XML — standard W3C Pronunciation Lexicon Specification format:
Plain text — line-based format:
  • word=alias for alias items
  • word:/phoneme/ for IPA phonemes

In-Call Playback

Source: https://developers.telnyx.com/docs/voice/tts/in-call-playback.md
In-call TTS plays synthesized speech during live voice calls. Two integration paths:

Voice API

Use the speak command to play TTS on an active call:
See Voice API docs for the full command reference.

TeXML

Use the “ element:
See TeXML docs for the full “ reference.

AI Assistants

AI Assistants use TTS for voice output. Configure the voice model in assistant settings. See AI Assistants for voice configuration.

Voice Selection

In-call TTS uses the same voice format as WebSocket and REST:
All models (including Ultra) are available for in-call playback.

Pricing

Source: https://developers.telnyx.com/docs/voice/tts/rest-api/pricing.md
Pricing for WebSocket TTS varies by engine and model. Contact sales or check the pricing page for current rates.

Voice Design

Overview

Source: https://developers.telnyx.com/docs/voice/voice-design-lab.md
The Voice Design lets you create custom voices for text-to-speech. No recording studio, no training datasets, no waiting. Start from a description. Write what you want in plain text — age, tone, accent, energy — and the AI generates it. Start from a recording. Upload an audio clip (or record one in the browser) and the AI captures that voice identity.

Overview

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/design-voice/concepts.md

What voice design does

Voice design generates a synthetic voice from a natural language description. You describe what you want — age, tone, accent, pacing — and the AI creates audio samples that match. This is not voice cloning. There’s no source audio. The voice is generated from scratch based on your text prompt.

The two-step flow: design → clone

The API has two separate resources:
  1. Voice Design — an intermediate artifact. Think of it as a draft. You can iterate on it (up to 50 versions per design). It is NOT usable for TTS directly.
  2. Voice Clone — a production-ready voice. Created from a design. This is what you pass to AI Assistants, Call Control, and the TTS API.
The portal hides this two-step flow behind a single “Save This Voice” button. If you’re using the API directly, you need both steps.

Quickstart

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/design-voice/quickstart.md

Portal walkthrough

Select Telnyx or Minimax using the provider toggle. Write a natural language description of the voice you want — gender, age, tone, pace, texture, personality. Click Generate Samples to create three audio previews. Each reads a different script in your chosen language. Listen to each sample. Click Regenerate All to try again, or refine your description. Click Save This Voice. Give it a name and gender tag — this creates a production-ready voice clone.

Using the API

1. Create a voice design

Set "provider": "minimax" to use the Minimax provider instead.

2. Listen to the generated sample

Returns audio/wav.

3. Save as a usable voice clone

A voice design is a draft. To use it in production, save it as a clone:

Full example

Python
Node.js
Once saved, see Using Custom Voices for how to use it in AI Assistants, Call Control, and the TTS API.

Parameters

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/design-voice/api-details.md

Providers

Set via the provider body parameter on POST /v2/voice_designs.

Generation Parameters

Body parameters on POST /v2/voice_designs. Telnyx provider only — ignored when provider is "minimax". The defaults are a great starting point — you can skip these parameters entirely and get good results. Adjust them later if you want to fine-tune the output.

Prompting Guide

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/design-voice/prompting-guide.md
Structure your prompt for consistent results:
Example:
Female, mid-thirties. Warm and full, slightly husky. Moderate pace, sounds like someone who smiles while talking.

Dimensions to describe

Age

Tone / Timbre

  • Deep / low-pitched — gravitas, authority
  • Smooth / rich — polished, professional
  • Gravelly / raspy — character, authenticity
  • Airy / breathy — intimate, soft
  • Warm / mellow — approachable, friendly

Gender

Male, female, or describe the sound directly: “a lower-pitched, husky female voice” or “a neutral, mid-pitched androgynous voice.”

Pacing

  • Measured / deliberate — careful, authoritative
  • Rapid-fire / quick — energetic, urgent
  • Relaxed / conversational — natural, approachable
  • Rhythmic — storytelling, narration

Emotion / Energy

  • Calm / serene — support, meditation
  • Enthusiastic / upbeat — marketing, announcements
  • Authoritative / matter-of-fact — IVR, instructions
  • Warm / empathetic — customer service, healthcare

Accent / Regional

Describe the regional quality you want. Be specific:
  • “Slight British accent” rather than “British”
  • “Neutral American” rather than just “American”
  • “Soft Southern drawl” rather than “Southern”

Use case context

Adding context helps the model understand intent:
  • “Customer service agent for a bank”
  • “Podcast narrator for true crime”
  • “Bedtime story reader for children”

Example prompts

Common pitfalls

  • Too vague — “nice voice” or “good voice” produces generic output. Be specific about at least 3 dimensions.
  • Contradictory traits — “whisper” + “booming” confuses the model. Pick a coherent set of characteristics.
  • Provider differences — the same prompt may produce noticeably different results on Telnyx vs Minimax. Try both.
  • Ignoring the preview text — the text you provide for synthesis should match the voice’s intended use. Don’t use a cheerful script for a somber voice.

The Enhance button

The portal’s Enhance button uses AI to expand a short description into a detailed prompt. This is a good starting point, but review the expanded prompt before generating — you may want to tweak specific dimensions.

Overview

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/concepts.md

What voice cloning does

Voice cloning captures a speaker’s vocal characteristics — timbre, cadence, accent, pronunciation — from a short audio sample and applies them to new speech synthesis. The clone is a representation of the voice, not a recording of it. The system learns patterns from your audio and encodes them into parameters that guide TTS. This means:
  • The cloned voice can say things the original speaker never said
  • Clone quality is bounded by what the model can learn from your sample
  • Poor recordings, background noise, or inconsistent delivery degrade the clone

What cloning doesn’t do

A clone is not a recording. It’s a statistical approximation of a voice — the model extracts patterns (formant frequencies, prosodic tendencies, spectral characteristics) and applies them during synthesis. This means:
  • Output passes through the TTS model, which has its own characteristics. A clone sounds like the speaker, but through the lens of the model.
  • Quality has a ceiling set by your source audio. No amount of API parameters will fix a noisy or inconsistent recording.
  • The clone may not handle speech styles far from the original sample well. A voice cloned from calm narration may sound different when asked to express strong emotion.

Two ways to create a clone

Both produce the same output: a voice clone with a voice ID you can use in production.

Recording best practices

  1. Match your recording to your use case. Don’t read a monotone script if you want an expressive clone. The AI replicates what it hears — including energy, emotion, and pacing.
  2. Speak clearly, avoid background noise. Use a decent microphone in a quiet space. Background noise gets cloned too. You don’t need a 10Kmica10K mic — a 100-300 USB condenser in a quiet room is sufficient.
  3. Avoid long pauses. The cloned voice will mimic pauses between sentences. Keep speech flowing naturally.
  4. Trim your recording. Speech from start to finish, no dead air at the beginning or end.
  5. Speak in the target language. If you want the clone to speak Spanish, record in Spanish.
  6. Keep it consistent. Same tone, accent, and energy throughout. Wide fluctuations confuse the model. The AI clones everything — including stutters, “uhms”, and inconsistencies.
  7. Aim for the right volume. Target -23 to -18 dB RMS with peaks no higher than -3 dB. Too quiet = noise floor issues. Too loud = clipping.
  8. Audio codec doesn’t matter much. MP3 at 128 kbps or above is fine. WAV is ideal but higher bitrate MP3 won’t noticeably hurt quality.
  9. Optimal duration by model:
    • Qwen3TTS: 5–10 seconds. Auto-trims to 10s. More isn’t better.
    • Ultra: Up to 10 seconds.
    • Minimax: 1–2 minutes is the sweet spot. Longer recordings capture more vocal range, but beyond 3 minutes yields diminishing returns.

Quickstart

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/quickstart.md

Upload a file

Select Telnyx or Minimax using the provider toggle. In the Voice Design, click Upload Audio and choose your file or drag and drop it. Enter a name for the voice and select the gender. Click Clone Voice. The system processes the audio and creates a voice clone, typically in a few seconds.

Record directly in the browser

Click Upload Audio, then select the Record tab. Select the language you’ll speak in. The system generates a reading script optimized for voice cloning. Click Start Recording and read the script clearly. It’s designed to capture the full range of phonemes. Listen to your recording. Re-record if needed, then click Clone Voice.

Using the API

Clone with Telnyx (default)

Clone with Minimax

Supports longer audio (up to 5 minutes):

Clone with Ultra model

Ultra clones use the higher-quality Ultra model. The request returns 202 Accepted — poll the clone’s status until it becomes active.
Response (202 Accepted):
Poll with GET /v2/voice_clones until status is active.

SDK Examples

Clone with Telnyx (default)

Python
Node.js

Clone with Minimax

Minimax supports longer audio (up to 20MB) and uses the speech-2.8-turbo model.
Python
Node.js

Clone with Ultra model

Ultra clones return 202 Accepted and require polling until the status is active.
Python
Node.js
Once saved, see Using Custom Voices for how to use it in AI Assistants, Call Control, and the TTS API.

Parameters

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/parameters.md

Models

Set via model_id (body) on POST /v2/voice_clones/from_upload, or use provider (body) to select Minimax.

Audio Requirements

Body parameter audio_file (multipart) on POST /v2/voice_clones/from_upload.
  • Qwen3TTS: aim for 5–10 seconds. Longer isn’t better — auto-trims to 10s.
  • Minimax: longer is better. 1–2 minutes of varied speech gives more vocal range.

The ref_text Parameter

Body parameter on POST /v2/voice_clones/from_upload. Optional. A transcript of what’s being said in the audio. Improves clone quality by giving the model a text reference to align against.

Ultra Async Flow

When model_id is "Ultra", the API returns 202 Accepted instead of 201:
Poll until ready:
See Responses for status values and voice ID format. See Errors for Minimax error codes.

Responses

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/responses.md

Voice ID Format

Every clone response includes fields to construct the voice ID: &#123;Provider&#125;.&#123;Model&#125;.&#123;provider_voice_id&#125; See Using Custom Voices for how to use these IDs across products.

Clone Status

Qwen3TTS and Minimax clones are always active on creation.

Errors

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/errors.md

General Errors

These errors apply to all providers.

Telnyx / Cartesia Errors

Minimax Errors


Using Custom Voices

Source: https://developers.telnyx.com/docs/voice/voice-design-lab/using-custom-voices.md
Every voice clone gets a unique voice ID: &#123;Provider&#125;.&#123;Model&#125;.&#123;voice_id&#125;
  • Telnyx: Telnyx.Qwen3TTS.33226e69-3abd-429b-b64a-86775c9b5850
  • Minimax: Minimax.speech-2.8-turbo.TB4ZMVKanThGeldiw8rLBEg21v4ifjUTRgLpkodJxpMYV
Find it in the Voice Design by clicking on any saved voice, or build it from the clone response’s provider, provider_supported_models, and provider_voice_id fields.

AI Assistants

Select your custom voice in the assistant’s voice settings. Telnyx clones appear under Telnyx / Qwen3TTS, Minimax clones under Minimax.

Call Control

Pass the voice ID in the voice field of the speak command.

TTS WebSocket

Pass the voice ID as the voice query parameter on the WebSocket URL. See the TTS streaming guide for the full connection flow.

API Reference (Voice)

Audio

  • Transcribe speech to text: Transcribe speech to text. This endpoint is consistent with the OpenAI Transcription API and may be used with the OpenAI JS or Python SDK.

Text To Speech Commands

  • List available voices: Retrieve a list of available voices from one or all TTS providers. When provider is specified, returns voices for that provider only. Otherwise, returns voic…

Voice Designs

  • List voice designs: Returns a paginated list of voice designs belonging to the authenticated account.
  • Create or add a version to a voice design: Creates a new voice design (version 1) when voice_design_id is omitted. When voice_design_id is provided, adds a new version to the existing design instead…
  • Get a voice design: Returns the latest version of a voice design, or a specific version when ?version=N is provided. The id parameter accepts either a UUID or the design name.
  • Rename a voice design: Updates the name of a voice design. All versions retain their other properties.
  • Delete a voice design: Permanently deletes a voice design and all of its versions. This action cannot be undone.
  • Download voice design audio sample: Downloads the WAV audio sample for the voice design. Returns the latest version’s sample by default, or a specific version when ?version=N is provided. The `…
  • Delete a specific version of a voice design: Permanently deletes a specific version of a voice design. The version number must be a positive integer.

Voice Clones