Telnyx Voice — Full Documentation
Voice API, speech-to-text, text-to-speech, and voice design. Complete page content for the Voice section of the Telnyx developer docs (https://developers.telnyx.com). Root index: https://developers.telnyx.com/llms.txt · Lightweight index for this section: https://developers.telnyx.com/development/llms/voice-llms-txt.md
Overview
Overview
Source: https://developers.telnyx.com/docs/voice/overview.mdReal-time and file-based transcription with four engine options: Telnyx, Google, Deepgram, and Azure. Convert text to natural-sounding audio via WebSocket streaming. Multiple voice providers and languages. Clone voices from audio samples or design custom AI voices from natural language prompts.
STT
Overview
Source: https://developers.telnyx.com/docs/voice/stt/overview.mdTelnyx STT transcribes audio to text in three ways: Stream audio over a persistent WebSocket connection. Real-time partial and final transcripts. Upload audio files via REST API. OpenAI-compatible endpoint. Enable transcription during live voice calls via Call Control or TeXML.
Quickstart
Source: https://developers.telnyx.com/docs/voice/stt/getting-started.md
Prerequisites
- A Telnyx account — sign up here
- An API key — create one in the portal
main.py:
index.js:
Example output
base_url and api_key and your existing code works.
Install:
main.py:
index.js:
Example response
model="deepgram/nova-3" with response_format=verbose_json. The Whisper models (openai/whisper-large-v3-turbo, openai/whisper-tiny) return text only.
What’s next
- Models & Engines — pick the right engine and model for your use case
- WebSocket Parameters — interim results, keyword boosting, endpointing, redaction
- REST API Parameters — all request body fields for file-based transcription
- In-Call Transcription — enable STT during live voice calls
Models
Source: https://developers.telnyx.com/docs/voice/stt/models.md
Comparison
Engine Details
The default WebSocket engine. Best English accuracy and the richest feature set. For REST, you must explicitly setmodel="deepgram/nova-3" — the REST default is openai/whisper-large-v3-turbo.
Models:
nova-3— Latest and most accurate. Supports diarization, word-level timestamps, smart formatting, numerals, and punctuation viamodel_config. Use this unless you need the lowest possible latency.nova-2— Previous generation. Still supported but nova-3 is better in all benchmarks.flux— Purpose-built for voice agents. Lowest latency with built-in end-of-turn detection — tells you when the speaker has finished so your agent can respond. WebSocket only.
multi mode (10 languages with code-switching). Flux supports English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch. See Deepgram languages.
Telnyx runs Whisper models on-network.
Models:
openai/whisper-large-v3-turbo— Multilingual (50+ languages, auto-detected). Returns text only — no timestamps regardless of response format.openai/whisper-tiny— Lightweight, lowest resource usage.
auto_detect to skip the language hint. See the Whisper language list.
Limitations: No diarization. No word-level timestamps.
Google Cloud Speech-to-Text integration.
Model: latest_long
Languages: 125+ languages/locales. See Google Cloud STT languages.
Microsoft Azure Speech Services integration.
Model: azure/fast
Languages: 100+ languages/locales with strong accent and dialect coverage. See Azure Speech languages.
xAI Grok STT integration for real-time transcription.
Model: xai/grok-stt
Languages: 25 languages, including Arabic, English, French, German, Hindi, Japanese, Korean, Portuguese, Spanish, and Vietnamese.
AssemblyAI Universal-Streaming integration for real-time voice agent transcription.
Model: assemblyai/universal-streaming
Languages: English, Spanish, German, French, Portuguese, and Italian.
Speechmatics real-time transcription integration with high accuracy and multilingual support including bilingual packs.
Model: speechmatics/standard
Languages: English, Spanish, plus bilingual/multilingual packs including Arabic–English, Mandarin–English, English–Malay, English–Tamil, Tagalog, and Spanish–English bilingual. Also supports Basque, Galician, Irish, Maltese, Mongolian, Swahili, Uyghur, and Welsh.
Features: Supports interim results (partial transcripts) and graceful CloseStream shutdown.
Soniox real-time transcription integration with automatic language detection.
Model: soniox/stt-rt-v4
Languages: Automatic detection — no language hint required.
Features: Supports interim results (partial transcripts), endpointing, and graceful CloseStream shutdown.
Self-hosted NVIDIA Parakeet integration for multilingual transcription with automatic language detection.
Model: nvidia/parakeet-v3
Languages: Automatic detection — no language hint required.
Features: Final transcripts only — no interim/partial results. Ignores endpointing. Accepts linear16/linear32 (16 kHz) audio only.
How to Choose
Need the highest accuracy for English? → Deepgramnova-3 — best WER (word error rate) across all English variants.
Building a voice agent that needs to know when the user stopped talking?
→ Deepgram flux — lowest latency with built-in end-of-turn detection.
Need to transcribe files in 50+ languages?
→ Telnyx openai/whisper-large-v3-turbo via REST API.
Need diarization (who said what)?
→ Deepgram nova-3 with model_config.diarize: true.
Need broad accent/dialect support?
→ Azure azure/fast — strong coverage across regional accents.
Need Grok STT for real-time calls?
→ xAI xai/grok-stt via WebSocket or Voice API.
Need low-latency streaming for voice agents?
→ AssemblyAI assemblyai/universal-streaming via WebSocket or Voice API.
Need high-accuracy multilingual with bilingual packs?
→ Speechmatics speechmatics/standard via WebSocket or Voice API.
Need real-time transcription with automatic language detection?
→ Soniox soniox/stt-rt-v4 via WebSocket or Voice API.
Need self-hosted multilingual transcription with automatic language detection?
→ Parakeet nvidia/parakeet-v3 via WebSocket or Voice API.
Specifying the Engine and Model
WebSocket — set via query parameters:model body parameter:
Migration
Source: https://developers.telnyx.com/docs/voice/stt/migration.mdMost migrations require changing 2–3 lines of code. Pick your current provider below.
WebSocket
Response field mapping
REST
Response field mapping
WebSocket
Response field mapping
REST
Response shape: identical. Telnyx returns the same
{"text": "..."} shape — no parsing changes needed.
REST
The easiest migration — Telnyx REST is OpenAI SDK compatible. Change the API key and base URL, everything else stays the same.Python
JavaScript
Response shape: Telnyx returns the same
{"text": "..."} shape — no parsing changes needed for the default json response format.
Note on verbose_json: OpenAI’s Whisper API returns segments with timestamps when you set response_format=verbose_json. Telnyx’s Whisper models (openai/whisper-large-v3-turbo, openai/whisper-tiny) return text only — no segments. If you need timestamps, switch to model="deepgram/nova-3" which supports segment- and word-level timestamps via model_config.
WebSocket
Google uses gRPC with protobuf. Telnyx uses WebSocket with JSON — no protobuf compilation, no service account credentials.
What you drop: protobuf definitions, gRPC client setup, service account credentials.
Response field mapping
WebSocket
AWS Transcribe Streaming uses HTTP/2 with event streams via theamazon-transcribe-streaming-sdk. Telnyx uses a plain WebSocket — no AWS SDK, no SigV4 signing, no IAM credentials.
Response field mapping
WebSocket
The Azure Speech SDK wraps a region-specific WebSocket with SDK abstractions and event handlers. Telnyx uses a plain WebSocket — no SDK install, no region routing.
Response field mapping
What you drop: Azure Speech SDK install, region routing, event-handler boilerplate, subscription key management.
REST
Response field mapping
Lifecycle
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming.mdReal-time speech-to-text over a persistent WebSocket connection. Send audio, receive transcripts.
Endpoint
Connection Lifecycle
1. Handshake
The connection starts as an HTTP GET withUpgrade: websocket. The server responds with 101 Switching Protocols, then the connection upgrades to WebSocket frames.
2. Streaming
Once connected, audio and transcription flow concurrently — no request/response pairing. Client → Server
Server → Client
See Messages for the complete wire protocol reference.
3. Teardown
Send{"type": "CloseStream"} (Deepgram, Speechmatics, and Soniox) to flush remaining audio and close gracefully. The server finishes processing, sends any remaining transcripts, then closes the WebSocket.
CloseStream works but may lose buffered audio on Deepgram, Speechmatics, and Soniox.
See Examples for complete code samples.
Overview
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters.mdAll parameters are passed as query string values on the WebSocket URL. Locked at connection time — cannot be changed mid-session.
Deepgram, Telnyx, Google, Azure, xAI, AssemblyAI, Speechmatics, Soniox, Parakeet
See Engines & Models
See Audio Formats
Hz. Required for raw encodings (linear16, mulaw, alaw). Ignored for container formats. Invalid value returns error 40005.
Language code or auto-detect behavior varies by engine. See Language for details.
"true" for partial transcripts. See Interim Results for details.
Silence detection in ms for engines that support endpointing. "false" to disable. See Endpointing for details.
PII redaction. Deepgram only. See Redaction for details.
Comma-separated boost terms. Deepgram Nova-3/Flux. See Keyword Boosting for details.
Legacy keyword boosting with intensifiers. Deepgram Nova. See Keyword Boosting for details.
Azure Speech Services region.
Flux only. Confidence threshold for EndOfTurn events. See End-of-Turn Detection for details.
Flux only. Max silence before forcing EndOfTurn. See End-of-Turn Detection for details.
Flux only. Speculative EagerEndOfTurn threshold. Disabled by default. See End-of-Turn Detection for details.
Example
Audio Formats
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/audio-formats.mdSet via the
input_format query parameter. Audio is sent as binary WebSocket frames — chunked bytes, no base64, no JSON wrapping.
Container formats (mp3, webm, etc.) are self-describing: the server demuxes the byte stream and extracts encoding/sample rate from headers. Raw formats have no metadata, so you must set sample_rate explicitly.
Works for both real-time capture (microphone, MediaRecorder, telephony bridge) and file streaming (read a file in chunks, push through the socket).
Browser Capture
Output fromMediaRecorder or similar browser APIs. Container headers carry sample rate.
Telephony
Codecs from voice networks. Raw frames,sample_rate required.
Invalid sample rate returns error 40005.
Raw PCM
Uncompressed audio from microphones, processing pipelines, or SDKs.sample_rate required.
Invalid sample rate returns error 40005.
Recorded File
Pre-recorded files read in chunks and streamed through the socket. Container headers carry sample rate.Engine Compatibility
Unsupported format/engine combination returns error 40002. Unsupported Flux format returns error 40006. Deepgram has three model generations with different format support. Flux is the most restrictive of Deepgram’s models — it dropsmp3, flac, webm_opus, amr_nb, amr_wb, g729, and speex compared to Nova.
Universal formats (all engines and models):
wav, linear16.
Engines & Models
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/engines-and-models.mdSet via
transcription_engine and model query parameters.
Engines
Flux Model
Deepgram’s lowest-latency model with built-in end-of-turn detection. Designed for real-time voice agents. See Audio Formats for supported formats.End-of-Turn Detection
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/end-of-turn.mdDeepgram Flux only. These parameters return a 400 error on non-Flux models. Flux uses a confidence-based system to decide when a speaker has finished their turn. Confidence threshold (
0.5–0.9) for triggering an EndOfTurn event. Higher values require more certainty the speaker is done — fewer false positives but slightly more latency. Lower values respond faster but may cut speakers off mid-thought.
Confidence threshold (0.3–0.9) for triggering an early EagerEndOfTurn event. Not set by default — setting it enables eager mode. When fired, your agent can start generating a response speculatively. If the speaker resumes, a TurnResumed event cancels it. Must be ≤ eot_threshold. Lower values = earlier triggers, more false starts. Typical range: 0.3–0.5 for ~150–250 ms latency savings at the cost of ~50–70% more LLM calls.
Maximum silence in ms (500–10000) before forcing EndOfTurn regardless of confidence. Resets when speech resumes. Increase for speakers who pause frequently; decrease for rapid-fire Q&A.
Event Flow
Without eager mode (eot_threshold only):
eager_eot_threshold set):
Configuration Profiles
Default — balanced for general use:Language
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/language.mdBCP-47 language code. Default:
en-US.
Auto-Detection
Passmulti to enable automatic language detection. The aliases auto and auto_detect are silently mapped to multi.
Engine Support
Supported Languages
For most engines, Telnyx passes the language code directly without validation. The supported set depends on which engine you use. Speechmatics is the exception — Telnyx accepts shorthand codes for bilingual/multilingual packs and maps them to the provider’slanguage + domain configuration internally.
Common Codes
Interim Results
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/interim-results.mdDeepgram, Speechmatics, and Soniox. Other engines ignore this parameter. Controls whether the server sends partial (non-final) transcripts as speech is processed.
false. Pass "true" as a string.
interim_results=false (default) — Server sends only final transcripts. Each message has is_final: true. Lower message volume, higher latency per result.
interim_results=true — Server sends evolving partial transcripts as audio is processed, followed by a final. Partials have is_final: false and are replaced by the next message.
confidence: 0.0 — confidence is only meaningful on final results.
Endpointing
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/endpointing.mdDeepgram, xAI, Google, Speechmatics, and Soniox. Other engines ignore this parameter. Controls how long the engine waits after silence before finalizing an utterance. Soniox has a different valid range. When
transcription_engine=Soniox, this parameter maps to max_endpoint_delay_ms and must be between 500 and 3000 ms. Values outside that range are rejected. The default (100 ms) and the low-value examples below apply to Deepgram, xAI, Google, and Speechmatics only.
100 ms (not applicable to Soniox — Soniox endpointing is disabled unless a value in the 500–3000 ms range is provided).
Values
Trade-offs
Low values (50–100 ms) — Fast response. Utterances may split mid-sentence on short pauses. (Deepgram, xAI, Google, Speechmatics only — below Soniox minimum.) High values (300–1000 ms) — More complete sentences. Higher latency before finalization. Soniox range (500–3000 ms) — Minimum 500 ms. Use 500–800 ms for responsive turn detection, 1000–3000 ms for longer utterances with natural pauses. Disabled ("false") — No automatic splits. Use Finalize control messages to manually trigger boundaries, or rely on CloseStream for a single final transcript.
Interaction With Utterance End
When endpointing triggers, Deepgram sends the final transcript followed by an utterance end event (ifutterance_end_ms is configured server-side — currently 1000 ms).
Keyword Boosting
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/keyword-boosting.mdDeepgram only. Other engines ignore these parameters. Two parameters control keyword boosting. They target different Deepgram model generations.
keyterm — Nova-3 and Flux
Comma-separated list of terms to boost. Simple — no intensifiers.
keywords — Nova (Legacy)
Terms with optional intensity scores. Format: keyword:intensifier.
Which To Use
Examples
Boost multiple terms on Nova-3:Redaction
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/parameters/redaction.mdDeepgram only. Other engines ignore this parameter. Replaces sensitive data in transcripts with placeholder text.
Values
Multiple values can be passed as comma-separated:
Output
Redacted content is replaced in the transcript text. The exact replacement format depends on the Deepgram model.Messages
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/responses.mdThe WebSocket carries two frame types: binary frames (audio) from client to server, and JSON text frames in both directions.
Client → Server
Audio Data
Binary WebSocket frames containing raw audio bytes. No base64, no JSON wrapping. Recommended chunk size: 2048–8192 bytes. Smaller chunks reduce latency; larger chunks reduce round trips.Control Messages
JSON text frames with atype field.
Finalize
CloseStream
KeepAlive
Unknown text frames are silently ignored.
Server → Client
All server messages are JSON text frames.Transcription Result
Emitted for each recognized speech segment (partial or final).Utterance End
Emitted on speaker pause (Deepgram). Empty transcript,is_final: true.
Error
Emitted on validation or connection errors. Connection closes shortly after.Message Flow
interim_results=false (default) — server sends only final transcripts:
interim_results=true — server sends partials, then final:
is_final: true results are stable.
Errors
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/errors.mdIf you send an invalid parameter (unsupported engine, format, or format/engine combination), the server responds with a JSON error and closes the connection.
Error Response Format
Error Codes
Examples
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/examples.mdStream a WAV file and print transcripts.
Python
JavaScript
Production Patterns
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/production-patterns.mdUse these patterns when running the standalone WebSocket STT endpoint in production.
Connection Recovery
Treat the WebSocket session as disposable. Reconnect on network failure, server close, idle timeout, and process restart.
Set all query parameters on every reconnect. STT configuration cannot be changed mid-session.
Backoff
Use bounded exponential backoff with jitter.
Add random jitter of 0-500 ms per attempt. Reset the attempt counter after a stable connection.
Do not retry immediately on authentication or validation errors. Fix the API key, query parameters, engine, model, or format first.
Partials
Enableinterim_results=true when the application needs live captions or low-latency UI updates.
Store final transcript segments separately from the current partial. This prevents duplicate text when a final result arrives after one or more interim results.
Audio Buffering
Buffer audio at the producer boundary, not inside the WebSocket send loop.
Avoid unbounded queues. A slow or disconnected socket should not grow memory usage indefinitely.
For live audio, prefer dropping stale buffered audio over sending it late. Late audio increases transcript delay and can make captions appear out of sync.
Keepalive
For Deepgram sessions, send{"type": "KeepAlive"} during long silence periods. Keep sending audio as binary frames when audio is available.
For other engines, use the WebSocket client’s ping/pong support when available and reconnect on missed heartbeats.
Monitoring
Track connection, latency, transcript, and buffer metrics.
Log the selected
transcription_engine, model, input_format, sample_rate, and interim_results value with each session. Redact API keys and user audio.
Shutdown
Use graceful shutdown for planned stops.- Drain the audio queue.
- Send
{"type": "CloseStream"}. - Wait for final transcript messages.
- Close the WebSocket.
Pricing
Source: https://developers.telnyx.com/docs/voice/stt/websocket-streaming/pricing.mdPricing for WebSocket STT varies by engine and model. Contact sales or check the pricing page for current rates.
Overview
Source: https://developers.telnyx.com/docs/voice/stt/rest-api.md
POST /v2/ai/audio/transcriptions
Synchronous file transcription. Upload audio or pass a URL, get text back.
Feature Support
If you’re coming from alternative providers:Quick Start
Python
JavaScript
Overview
Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters.mdAll parameters are sent as
multipart/form-data.
Model to use for transcription. See Models for details.
Values: openai/whisper-large-v3-turbo (default), openai/whisper-tiny, deepgram/nova-3
Audio file to transcribe. Mutually exclusive with file_url. See Audio Formats for supported formats, size limits, and per-model restrictions.
Publicly accessible URL to an audio file. Mutually exclusive with file. See Audio Formats for details on how file and file_url differ.
Language hint. Behavior varies by model — see Language.
Output shape. See Response Format.
Values: json (default), verbose_json
Timestamp detail level. Only valid with response_format=verbose_json — returns 400 otherwise.
Values: segment
Deepgram-specific options. Only valid with deepgram/nova-3 — returns 400 for other models. See Model Config.
Models
Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters/models.mdYour choice of
model determines which audio formats are accepted, what language values are valid, and what response fields are available.
openai/whisper-large-v3-turbo
Default model. Multilingual. Auto-detected if language omitted. See Whisper docs for the full language list. Returns text only — no timestamps regardless of response_format.
openai/whisper-tiny
Lightweight, lowest resource usage. Multilingual (50+ languages, auto-detected). Returns text only — no timestamps.
deepgram/nova-3
Highest accuracy for English. Advanced features (diarization, word timestamps, smart formatting, numerals, punctuation) available via model_config. Defaults language to en if omitted. Can also set language inside model_config — top-level field takes precedence. See Deepgram language docs for details.
Audio Formats
Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters/audio-formats.mdApplies to both
file (multipart upload) and file_url (URL download).
Common
- Max size: 100 MB
- Processing: All audio is decoded, resampled to 16kHz, and mixed to mono via ffmpeg before transcription. Container format doesn’t matter as long as ffmpeg can decode it — the validated extension list is the actual restriction.
Supported Formats
file vs file_url
One of
file or file_url is required. Sending both returns 400.
Model Config
Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters/model-config.mdDeepgram only. Returns 400 if used with other models. Pass-through to Deepgram’s pre-recorded API query parameters. Every key-value pair in
model_config is forwarded directly — Telnyx does not validate individual options.
Commonly Used Options
Example
model_config can be sent as a JSON string in the multipart form field. The server parses it before forwarding.
Unvalidated Pass-Through
Any Deepgram query parameter can be passed. If Deepgram adds new options, they work immediately without a Telnyx API update. Conversely, invalid keys are forwarded and may cause Deepgram to return an error. Refer to Deepgram’s API reference for the full list of supported parameters.Response Format
Source: https://developers.telnyx.com/docs/voice/stt/rest-api/parameters/response.mdControlled by the
response_format parameter.
json (Default)
Text only.
verbose_json
Adds duration (seconds) and timestamped segments — only when using deepgram/nova-3. The Whisper models (openai/whisper-large-v3-turbo, openai/whisper-tiny) return text only regardless of response_format. See the Timestamp Availability table below.
Example response with model=deepgram/nova-3:
timestamp_granularities[]=segment alongside response_format=verbose_json. Using timestamp_granularities without verbose_json returns 400.
Segment Fields
Timestamp Availability by Model
Streaming Response (Undocumented)
SendingAccept: application/stream+json returns newline-delimited JSON chunks as segments are transcribed. Each line:
Pricing
Source: https://developers.telnyx.com/docs/voice/stt/rest-api/pricing.mdPricing for REST API STT varies by engine and model. Contact sales or check the pricing page for current rates.
In-Call Transcription
Source: https://developers.telnyx.com/docs/voice/stt/in-call-transcription.mdIn-call transcription enables real-time STT on live voice calls. The audio codec is managed by the Telnyx platform — no format configuration needed. Two integration paths:
- Voice API —
transcription_start/transcription_stopcommands on active calls. See the Voice API STT guide. - TeXML — XML-based call flow with transcription directives. See the TeXML transcription guide.
TTS
Overview
Source: https://developers.telnyx.com/docs/voice/tts/overview.md
1. Choose your interface
Real-time streaming. Send text, receive audio chunks as they’re synthesized. HTTP POST. Get audio back as binary, base64, or async URL. OpenAI SDK compatible. TTS during live calls via Call Controlspeak or TeXML “.
2. Choose a pre-built voice
Natural, NaturalHD, Ultra, Kokoro, Qwen3TTS, xAI Grok. AWS Polly, Azure, ElevenLabs, Minimax, MurfAI, Rime, Resemble, Inworld, Fish Audio.3. Or create your own
Clone and design custom voices. Available on select providers: Qwen3TTS, Minimax, ElevenLabs, Resemble.Lifecycle
Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming.mdReal-time text-to-speech over a persistent WebSocket connection. Send text, receive audio.
Endpoint
Connection Lifecycle
1. Handshake
There are two ways to establish a connection:Direct WebSocket connection
You can connect directly to the WebSocket endpoint by passing all configuration as query parameters in thewss:// URL:
HTTP upgrade
Alternatively, initiate the connection as an HTTP GET request that upgrades to a WebSocket via the standard101 Switching Protocols handshake. This is what happens under the hood when a WebSocket client connects, and may be relevant if you need fine-grained control over the upgrade (e.g., setting custom headers in environments where the WebSocket library doesn’t expose them directly).
Initialization frame
Regardless of how the connection is established, send an initialization frame before any text:voice_settings to configure provider-specific parameters:
2. Streaming
Once initialized, text and audio flow concurrently — no request/response pairing. Text is buffered and synthesized at sentence boundaries. Client → Server
Server → Client
See Messages for the complete wire protocol reference.
"flush": true to force synthesis of buffered partials.
Text preprocessing: Markdown formatting is automatically stripped before synthesis (headers, bold, italics, code blocks, links, lists, emoji). Useful when synthesizing LLM output. Pronunciation dictionary replacements are applied if pronunciation_dict_id is set.
Streamed vs. concatenated delivery: Most providers (Telnyx Natural/NaturalHD/Qwen3TTS, Rime, Minimax, Resemble, Inworld) stream audio in separate frames where text is null. AWS Polly and Azure return audio in the text-bearing chunk instead. See Messages for details.
3. Teardown
Send{"text": ""} (empty string) to flush remaining buffered text and close gracefully. The server finishes synthesis, sends any remaining audio and a final frame, then closes the WebSocket.
Configuration
Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming/configuration.md
Two Configuration Surfaces
Both are one-shot. After the init frame, no configuration can change for the session. To change settings, open a new connection.
Query Parameters
Set on the URL at connect time. Immutable for the session.Voice Selection
The
voice_id segment (third part of the voice string) refers to different things depending on the provider and model:
The Voices API (
GET /v2/ai/tts/voices) returns all voices available to your account — pre-built and cloned — with each voice’s compound id ready to use as the voice parameter.
Connection Options
|
disable_cache | boolean | false | Bypass the audio cache and always synthesize fresh. |
Example
Voice Settings
Provider-specific tuning (speed, pitch, format, emotion, etc.) is not set via query parameters. It is passed once in thevoice_settings object on the initialization frame:
Messages
Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming/messages.md
Client → Server
All client messages are JSON text frames.Text Frame
Text to synthesize." " (single space) for handshake. "" (empty string) for end-of-sequence.
Provider-specific voice configuration. Only used in the handshake frame ({"text": " "}). See Voice Settings.
When true, immediately synthesizes all buffered text without waiting for a sentence boundary. Default: false.
When true, stops the current synthesis worker and starts a new one. The original handshake is replayed automatically. Use for barge-in/interruption.
Message Sequence
1. Handshake (required first message):Server → Client
All server messages are JSON text frames.Audio Chunk
Returned when synthesis produces audio for a complete sentence.null when the provider uses streamed delivery — audio arrives in separate streamed chunk frames instead. See note below.
The text segment this audio corresponds to. null for streamed audio chunks.
false for audio chunks.
true if audio was served from cache.
Time in milliseconds from speech request to first audio frame. Only present on the first chunk of each synthesis.
Streamed Audio Chunk
For providers that stream audio incrementally (Telnyx Natural, NaturalHD, Qwen3TTS, Rime, Minimax, Resemble, Inworld), audio arrives in separate frames:text is always null). The concatenated audio chunk for these providers has audio: null — only the streamed chunks carry audio bytes.
For AWS Polly and Azure, audio is returned in the audio field of the regular audio chunk frame. For all other providers, ignore the audio field on the text-bearing chunk and collect audio from the streamed frames.
Final Frame
Signals that synthesis is complete for the current text input:Error Frame
Errors
Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming/errors.md
HTTP Errors (Handshake)
These occur during the WebSocket upgrade request, before the connection is established.WebSocket Errors (Runtime)
After the connection is established, errors arrive as JSON frames:Error Messages
Troubleshooting
Examples
Source: https://developers.telnyx.com/docs/voice/tts/websocket-streaming/examples.md
Basic Streaming
Python
JavaScript
Conversational (Barge-In)
Send multiple text segments and interrupt mid-synthesis:LLM Token Streaming
Stream tokens from an LLM directly to TTS. The server buffers text and synthesizes at sentence boundaries:Overview
Source: https://developers.telnyx.com/docs/voice/tts/rest-api.md
How It Works
You send text in, audio streams back over the same HTTP connection. No polling, no callbacks. The response uses HTTP chunked transfer encoding — audio chunks arrive as they’re synthesized. Your client can begin playback immediately without waiting for the full file. The connection stays open until synthesis completes or 30 seconds pass with no new chunks. This makes REST suitable for real-time playback, not just batch file generation. For multi-turn conversational use cases where you’re continuously feeding text, use WebSocket Streaming instead.Text Preprocessing
Before synthesis, text passes through two stages:- Markdown stripping — headers, bold, italics, code blocks, links, lists, emoji are converted to plain text.
- Pronunciation dictionary — if
pronunciation_dict_idis set, custom word replacements are applied.
API Reference
The full OpenAPI spec for these endpoints is available in the auto-generated API Reference. Note: the OAS is currently being cleaned up — some fields and provider-specific schemas may be incomplete.Request
Source: https://developers.telnyx.com/docs/voice/tts/rest-api/request.md
Endpoint
Example
Request Body
Response
Source: https://developers.telnyx.com/docs/voice/tts/rest-api/response.mdThe
output_type request field controls what comes back.
Streaming Audio (default)
Withoutput_type: "binary_output" (or omitted), the response is raw audio over HTTP chunked transfer encoding:
Base64
Withoutput_type: "base64_output", the full audio is returned as a JSON payload after synthesis completes:
Async (audio_id)
Withoutput_type: "audio_id", synthesis runs in the background. You get a URL back immediately:
GET /v2/text-to-speech/speech/:audio_id. If the audio is still synthesizing, the GET response itself streams chunks as they become available.
Examples
Source: https://developers.telnyx.com/docs/voice/tts/rest-api/examples.md
OpenAI SDK Compatibility
The REST endpoint is a drop-in replacement for the OpenAI Audio API:API Reference
Source: https://developers.telnyx.com/docs/voice/tts/rest-api/api-reference.mdSee the full API reference: Generate Speech from Text.
Overview
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx.md
Selecting Telnyx
Telnyx is the default provider. If you don’t specify a provider, you get Telnyx. WebSocket:Models
Ultra is REST-only, not available over public WebSocket. Grok is available for Voice AI Assistants and direct REST TTS calls.
Browse all available voices via the Voices API or the Voice Design.
Natural
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/natural.mdVoice format:
Telnyx.Natural.
Pre-built English voices backed by Rime Mist.
Voice Samples
WebSocket
Query Parameters
Voice Settings
Send in the init frame ({"text": " "}):
REST API
Fields
Response
Default (binary_output): chunked audio bytes with Content-Type: audio/mpeg (or audio/wav, audio/pcm).
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
NaturalHD
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/naturalhd.mdVoice format:
Telnyx.NaturalHD.
Pre-built voices backed by Rime Arcana. 9 languages: en, fr, de, es, ar, hi, ja, he, pt.
Voice Samples
WebSocket
Query Parameters
Voice Settings
Send in the init frame ({"text": " "}):
REST API
Fields
Response
Default (binary_output): chunked audio bytes with Content-Type: audio/mpeg (or audio/wav, audio/pcm).
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
KokoroTTS
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/kokoro.mdVoice format:
Telnyx.KokoroTTS.
Lightweight, lowest-latency model. 5 languages: en, es, fr, it, pt.
Voice Samples
WebSocket
Query Parameters
Voice Settings
None. All synthesis parameters are fixed. The init frame only needs{"text": " "}.
REST API
Fields
No model-specific fields. Audio format is always MP3.Response
Default (binary_output): chunked audio bytes with Content-Type: audio/mpeg.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
Qwen3TTS
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/qwen3.mdVoice format:
Telnyx.Qwen3TTS.
Voice cloning model. 11 languages: en, zh, fr, de, it, ja, ko, pt, ru, es, ar.
The voice_id is the name of a clone you created in the Voice Design. Clones are scoped to your organization.
Voice Samples
WebSocket
Query Parameters
Voice Settings
Send in the init frame ({"text": " "}):
REST API
Fields
Response
Default (binary_output): chunked PCM audio bytes. Always 24kHz signed 16-bit LE mono.
With output_type: "base64_output": JSON with base64-encoded PCM.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
Ultra
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/ultra.mdVoice format:
Telnyx.Ultra.
Sub-100ms latency. 36 languages.
REST only — Ultra is not available over public WebSocket.
Voice Samples
SSML Emotions
Ultra supports inline SSML emotion tags. Place the tag before the text:angry, excited, content, sad, scared.
Additional: happy, enthusiastic, curious, calm, grateful, affectionate, sarcastic, surprised, confident, hesitant, apologetic, determined, frustrated, disappointed.
Omitting the tag = neutral delivery. Use sparingly — Ultra interprets emotional subtext from the text itself.
Nonverbal Cues
Insert[laughter] inline for natural laughing:
Language Support
Setlanguage_boost to improve pronunciation for the target language:
Arabic, Bengali, Bulgarian, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Gujarati, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Marathi, Māori, Norwegian, Polish, Portuguese, Punjabi, Romanian, Russian, Slovak, Spanish, Swedish, Tamil, Telugu, Thai, Turkish, Ukrainian, Vietnamese.
REST API
Fields
Response
Default (binary_output): chunked audio bytes with Content-Type: audio/mpeg.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
See also
xAI Grok is the second TTS provider supporting Expressive Mode. For Grok voice options, see Grok Voices. Note: Grok voices have higher latency than Ultra.Grok
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/grok.mdxAI Grok voices are expressive text-to-speech voices for Voice AI Assistants. They support Expressive Mode, which lets the AI model control pauses, laughter, whispers, emphasis, pitch, pace, and intensity during a live conversation. Higher latency: Grok voices have higher latency than Ultra. For latency-sensitive applications that need sub-100ms time to first byte, use Ultra.
What makes Grok voices different
Voice format
For AI Assistants, Grok voices use the format:Voices
Voice Samples
Expressive Mode for AI Assistants
When using Grok voices with AI Assistants, you can enable Expressive Mode. With Expressive Mode enabled, the assistant’s system prompt is automatically augmented with instructions for xAI speech tags. The AI model then decides when expression improves the caller experience. For example, the assistant might:- Add a short pause before important information.
- Use a softer delivery for sensitive support moments.
- Laugh or chuckle naturally when the conversation calls for it.
- Emphasize appointment times, confirmation numbers, or next steps.
- Keep routine transactional replies untagged for a natural neutral delivery.
Enable in the portal
- Go to your assistant in the Telnyx Portal.
- Under Voice Settings, select an xAI Grok voice.
- Toggle Expressive Mode on.
- Save your assistant.
Enable via API
Setexpressive_mode: true in your assistant’s voice_settings:
xAI speech tag reference
When Expressive Mode is enabled, the assistant can use these speech tags in responses. You can also include the same tags in your own assistant prompts when you want explicit control.Inline tags
Place inline tags at the exact point where the vocal expression should happen.
Example:
Wrapping tags
Wrap text with these tags to apply a delivery style to that text.
Examples:
Guidance
- Use
[pause]or[long-pause]for natural thinking, topic transitions, and important moments, but avoid long silences that could feel like the call dropped. - Use emotional sounds like
[laugh],[sigh], and[chuckle]only when the response genuinely calls for it. - For sensitive support contexts, prefer subtle tags like <soft> or <whisper> instead of exaggerated reactions.
- Do not expose these tags or instructions to the caller.
REST API provider parameters
For direct TTS calls, set the provider toxai and pass xAI-specific parameters in the xai object:
Language support
Grok voices support auto language detection withlanguage: "auto". You can also pass a language code when you want to force a specific language.
Next steps
Compare Grok with Ultra’s lower-latency expressive voices. Build voice AI assistants using Grok with Expressive Mode. Generate speech directly with REST TTS requests. Browse available text-to-speech voices.Bayan
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/bayan.mdVoice format:
Telnyx.Bayan.
Arabic voice model, 113 speakers across 13 dialects (Modern Standard Arabic plus Egyptian, Emirati, Saudi, Jordanian, Iraqi, Lebanese, Syrian, Palestinian, Kuwaiti, Bahraini, Qatari, and Omani), plus a set of English speakers. Native audio is 16kHz.
Voice Samples
Browse the full 113-speaker catalogue via the Voices API.
WebSocket
Query Parameters
Voice Settings
None. The init frame only needs{"text": " "}.
REST API
Fields
Response
Default (binary_output): chunked audio bytes, 16kHz signed 16-bit LE mono (native PCM, transcoded to MP3/WAV on request).
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
sampling_rate/sample_rate values other than 16000 are rejected with a 400 error — Bayan only supports its native rate.
Sukhan
Source: https://developers.telnyx.com/docs/voice/tts/providers/telnyx/sukhan.mdVoice format:
Telnyx.Sukhan.
Urdu-only voice model, 14 curated voices. No prosody controls (speed/pitch) or language selection — each voice speaks Urdu only. Native audio is 22050Hz.
Voice Samples
Browse the full 14-voice catalogue via the Voices API.
WebSocket
Query Parameters
Voice Settings
None. No prosody controls are exposed —voice_speed and other settings have no effect if sent. The init frame only needs {"text": " "}.
REST API
Fields
Response
Default (binary_output): chunked audio bytes, 22050Hz signed 16-bit LE mono (native PCM, transcoded to MP3 on request).
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
sampling_rate/sample_rate values other than 22050 are rejected with a 400 error. Unsupported format values are also rejected — REST response_format accepts only pcm/mp3; WebSocket audio_format accepts only mp3/linear16 (its PCM equivalent). Neither accepts wav.
Rime
Source: https://developers.telnyx.com/docs/voice/tts/providers/rime.md
Models
Voice Format
Voice Samples
WebSocket
Query Parameters
Voice Settings
REST API
Fields
Response
Default (binary_output): chunked audio bytes.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
Minimax
Source: https://developers.telnyx.com/docs/voice/tts/providers/minimax.mdVoice format:
minimax..
voice_id can be a system voice (pre-built) or a cloned voice from the Voice Design (organization-scoped).
Voice Samples
WebSocket
Query Parameters
Voice Settings
REST API
Fields
Response
Default (binary_output): chunked audio bytes.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
Resemble
Source: https://developers.telnyx.com/docs/voice/tts/providers/resemble.mdVoice format:
resemble.Turbo.
Default model: Turbo. voice_id is a voice from your own Resemble account.
Voice Samples
WebSocket
Query Parameters
Voice Settings
REST API
Fields
Response
Default (binary_output): chunked audio bytes.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
Inworld
Source: https://developers.telnyx.com/docs/voice/tts/providers/inworld.mdVoice format:
inworld..
Models:
inworld-tts-1.5-mini(aliasMini) — faster, lower latency.inworld-tts-1.5-max(aliasMax) — higher quality.inworld-tts-2(aliasTTS2) — latest generation; supports thedelivery_modeparameter.
inworld-tts-1.5-mini if model omitted.
Voice Samples
WebSocket
Query Parameters
Voice Settings
REST API
Fields
Response
Default (binary_output): chunked audio bytes.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
xAI
Source: https://developers.telnyx.com/docs/voice/tts/providers/xai.mdVoice format:
xAI.
xAI Grok voices are expressive, multilingual text-to-speech voices. They support inline speech tags for pauses, vocal sounds, emphasis, pitch, pace, and intensity.
xAI Grok voices are higher-latency than Telnyx Ultra. For latency-sensitive applications that need sub-100ms time to first byte, use Ultra.
Voices
Voice Samples
WebSocket
xAI Grok voices are not available on the public TTS WebSocket API. Use the REST API for direct text-to-speech generation, or use xAI Grok voices with AI Assistants.REST API
Fields
Response
Default (binary_output): chunked audio bytes.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
Expressive speech tags
Use speech tags inline intext when you want more expressive delivery.
AI Assistants
For AI Assistants, choose an xAI Grok voice such asxAI.eve and enable Expressive Mode to let the assistant decide when speech tags improve the caller experience.
Build voice AI assistants using xAI Grok voices with Expressive Mode.
Generate speech directly with REST TTS requests.
Fish Audio
Source: https://developers.telnyx.com/docs/voice/tts/providers/fishaudio.mdVoice format:
FishAudio..
Fish Audio is a Telnyx-hosted external TTS provider. Telnyx exposes a hand-vetted shortlist of Fish Audio voices from their public Voice Library.
Fish Audio is cross-lingual — any voice can speak any language from the input text. The language field in the voice listing is the voice’s native accent only, not a synthesis constraint.
Models
All curated voices are available under all three models.
Curated Voices
Only curated catalog voices are accepted for synthesis. Arbitrary Fish Voice-Library
reference_ids cannot be used under the Telnyx API key.
Emotion Tags
Fish Audio S2 models (s2.1-pro, s2-pro) support inline emotion, tone, and audio-effect markers placed directly in the input text. S1 uses parentheses — (happy) — instead of brackets. The markers are processed by the upstream Fish Audio API and do not add latency or count toward token limits.
text field. Markers can be combined for layered effects — [excited][laughing] We won! Ha ha! — and accept intensity modifiers like [slightly sad], [very excited], [extremely angry].
For the full list of supported emotions, tone markers, audio effects, and S1 syntax, see Fish Audio’s Emotion Control guide.
WebSocket
Query Parameters
Voice Settings
Fish Audio does not expose speed or pitch controls.voice_settings are limited to format and sample rate overrides. Emotion and tone are controlled inline in the text via bracket markers — see Emotion Tags below.
REST API
Fields
Response
Default (binary_output): chunked audio bytes.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
Audio Formats & Sample Rates
Each format has a fixed set of accepted sample rates:
On WebSocket, the
audio_format query parameter and voice_settings.format are independent parameters with different vocabularies. The audio_format query param accepts mp3, wav, and linear16 (which maps to pcm). The voice_settings.format field accepts mp3, wav, pcm, and opus directly. If both are set, voice_settings.format takes precedence.
In-Call Playback
Fish Audio is available for in-call TTS via Call Controlspeak and TeXML “. For telephony playback, the gateway automatically delivers MP3 at 44.1 kHz to ensure correct resampling by the call-control audio pipeline.
See In-Call Playback for details.
Provider Docs
AWS Polly
Source: https://developers.telnyx.com/docs/voice/tts/providers/aws.mdVoice format:
aws.Polly..
Example: aws.Polly.Generative.Lucia
The engine can also be parsed from a hyphenated suffix on the voice ID — e.g., Lucia-longform resolves to engine long-form.
Voice Samples
WebSocket
Query Parameters
Voice Settings
REST API
Fields
Response
Default (binary_output): chunked audio bytes. Format depends on output_format.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
SSML Tags
Source: https://developers.telnyx.com/docs/voice/tts/providers/aws/ssml-tags.mdIn this tutorial, you’ll learn about SSML tags that can help customize your audio response in your text-to-speech application. This tutorial assumes you’ve already set up your developer account and environment and you know how to send commands and receive webhooks using the Telnyx Voice API.
What are SSML tags?
Speech Synthesis Markup Language (SSML) is an XML-based markup language that is used to generate synthetic speech for appliations. SSML tags are used to change the tone of speech in the application by adjusting pitching, volume, duration of speech, and more.SSML tag examples
Adding a pause
SSML Tag:
There are 2 ways for defining the length of the pause by using the following attributes:
- Time: Defines the number of s or ms
- Strength: Chooses the strength using the following values:
- None: no pause
- Pause: the same duration as after a period
- x-weak: the same as none.
- weak: sets a pause of the same duration as the pause after a comma
- medium: has the same strength as weak
- strong: sets a pause of the same duration as the pause after a sentence
- x-strong: sets a pause of the same duration as the pause after a paragraph.
Emphasizing words
SSML Tag:- Strong - increases the volume and slows the speaking rate
- Moderate - increases the volume and slows the speaking rate, but less than Strong
- Reduced - decreases the volume and speeds up the speaking rate
Set a different language
SSML Tag:Adding a pause between paragraphs
SSML Tag:Using phonetic pronunciation
SSML Tag:- Alphabet, with the following options:
- ipa, meaning the International Phonetic Alphabet (IPA) will be used
- x-sampa, which indicates that the Extended Speech Assessment Methods Phonetic Alphabet (X-SAMPA) will be used.
- ph, specifies how the text should be pronounced.
Controlling volume, speaking rate, and pitch
SSML Tag::- Volume:
- default: resets the volume to default value
- silent, x-soft, soft, medium, loud, x-loud: sets the volume to predefined value
- +ndB, -ndB: changes the volume relative to the current level
- Rate:
- x-slow, slow, medium, fast,x-fast: sets the pitch to a predefined value
- n%: a percentage change in speaking pace.
Adding a pause between sentences
SSML Tag:Controlling how special words are spoken
SSML Tags:- characters or spell-out
- cardinal or number
- digits
- fraction
- unit
- date
- time
- address
- telephone.
Pronouncing acronyms and abbreviations
SSML Tag:Azure
Source: https://developers.telnyx.com/docs/voice/tts/providers/azure.mdVoice format:
azure.
Example: azure.en-US-AvaMultilingualNeural
No model ID segment — Azure voices are flat identifiers. Default voice: en-US-AvaMultilingualNeural.
Voice Samples
WebSocket
Query Parameters
Voice Settings
REST API
Fields
Response
Default (binary_output): chunked audio bytes.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
ElevenLabs
Source: https://developers.telnyx.com/docs/voice/tts/providers/elevenlabs.mdElevenLabs requires your own API key configured in your Telnyx account. Telnyx relays requests to the ElevenLabs API — voice settings are passed through directly. Voice format:
elevenlabs..
Example: elevenlabs.v3.Adam
voice_id is a voice from your own ElevenLabs account — pre-built, cloned, or designed. Preview voices at elevenlabs.io/voice-library.
WebSocket
Query Parameters
Voice Settings
Relayed directly to the ElevenLabs API. Any field ElevenLabs accepts can be passed here.REST API
Fields
Response
Default (binary_output): chunked audio bytes. Format determined by ElevenLabs.
With output_type: "base64_output": JSON with base64-encoded audio.
With output_type: "audio_id": JSON with an audio_url for deferred retrieval.
Pronunciation Dictionaries
Source: https://developers.telnyx.com/docs/voice/tts/pronunciation-dictionaries.mdPronunciation dictionaries let you control how specific words and phrases are spoken during text-to-speech synthesis. Dictionaries are applied automatically before speech generation — no changes to your text input required.
Item Types
Each dictionary contains up to 100 items. Two types are supported:Alias (text replacement)
Replaces matched text with alternative text before synthesis:Phoneme (IPA notation)
Specifies exact pronunciation using the International Phonetic Alphabet:Using a Dictionary
Pass the dictionary ID when synthesizing speech: REST API:pronunciation_dict_id as a query parameter on the connection URL:
Managing Dictionaries
Create a dictionary
multipart/form-data instead of providing items as JSON. Plain text format:
List dictionaries
Get a dictionary
Update a dictionary
409 Conflict. Re-fetch and retry.
Delete a dictionary
Limits
File Upload Formats
When creating a dictionary via file upload, two formats are supported: PLS/XML — standard W3C Pronunciation Lexicon Specification format:word=aliasfor alias itemsword:/phoneme/for IPA phonemes
In-Call Playback
Source: https://developers.telnyx.com/docs/voice/tts/in-call-playback.mdIn-call TTS plays synthesized speech during live voice calls. Two integration paths:
Voice API
Use thespeak command to play TTS on an active call:
TeXML
Use the “ element:AI Assistants
AI Assistants use TTS for voice output. Configure the voice model in assistant settings. See AI Assistants for voice configuration.Voice Selection
In-call TTS uses the same voice format as WebSocket and REST:Pricing
Source: https://developers.telnyx.com/docs/voice/tts/rest-api/pricing.mdPricing for WebSocket TTS varies by engine and model. Contact sales or check the pricing page for current rates.
Voice Design
Overview
Source: https://developers.telnyx.com/docs/voice/voice-design-lab.mdThe Voice Design lets you create custom voices for text-to-speech. No recording studio, no training datasets, no waiting. Start from a description. Write what you want in plain text — age, tone, accent, energy — and the AI generates it. Start from a recording. Upload an audio clip (or record one in the browser) and the AI captures that voice identity.
Overview
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/design-voice/concepts.md
What voice design does
Voice design generates a synthetic voice from a natural language description. You describe what you want — age, tone, accent, pacing — and the AI creates audio samples that match. This is not voice cloning. There’s no source audio. The voice is generated from scratch based on your text prompt.The two-step flow: design → clone
The API has two separate resources:- Voice Design — an intermediate artifact. Think of it as a draft. You can iterate on it (up to 50 versions per design). It is NOT usable for TTS directly.
- Voice Clone — a production-ready voice. Created from a design. This is what you pass to AI Assistants, Call Control, and the TTS API.
Quickstart
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/design-voice/quickstart.md
Portal walkthrough
Select Telnyx or Minimax using the provider toggle. Write a natural language description of the voice you want — gender, age, tone, pace, texture, personality. Click Generate Samples to create three audio previews. Each reads a different script in your chosen language. Listen to each sample. Click Regenerate All to try again, or refine your description. Click Save This Voice. Give it a name and gender tag — this creates a production-ready voice clone.Using the API
1. Create a voice design
"provider": "minimax" to use the Minimax provider instead.
2. Listen to the generated sample
audio/wav.
3. Save as a usable voice clone
A voice design is a draft. To use it in production, save it as a clone:Full example
Python
Node.js
Parameters
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/design-voice/api-details.md
Providers
Set via theprovider body parameter on POST /v2/voice_designs.
Generation Parameters
Body parameters onPOST /v2/voice_designs. Telnyx provider only — ignored when provider is "minimax".
The defaults are a great starting point — you can skip these parameters entirely and get good results. Adjust them later if you want to fine-tune the output.
Prompting Guide
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/design-voice/prompting-guide.md
Recommended format
Structure your prompt for consistent results:Female, mid-thirties. Warm and full, slightly husky. Moderate pace, sounds like someone who smiles while talking.
Dimensions to describe
Age
Tone / Timbre
- Deep / low-pitched — gravitas, authority
- Smooth / rich — polished, professional
- Gravelly / raspy — character, authenticity
- Airy / breathy — intimate, soft
- Warm / mellow — approachable, friendly
Gender
Male, female, or describe the sound directly: “a lower-pitched, husky female voice” or “a neutral, mid-pitched androgynous voice.”Pacing
- Measured / deliberate — careful, authoritative
- Rapid-fire / quick — energetic, urgent
- Relaxed / conversational — natural, approachable
- Rhythmic — storytelling, narration
Emotion / Energy
- Calm / serene — support, meditation
- Enthusiastic / upbeat — marketing, announcements
- Authoritative / matter-of-fact — IVR, instructions
- Warm / empathetic — customer service, healthcare
Accent / Regional
Describe the regional quality you want. Be specific:- “Slight British accent” rather than “British”
- “Neutral American” rather than just “American”
- “Soft Southern drawl” rather than “Southern”
Use case context
Adding context helps the model understand intent:- “Customer service agent for a bank”
- “Podcast narrator for true crime”
- “Bedtime story reader for children”
Example prompts
Common pitfalls
- Too vague — “nice voice” or “good voice” produces generic output. Be specific about at least 3 dimensions.
- Contradictory traits — “whisper” + “booming” confuses the model. Pick a coherent set of characteristics.
- Provider differences — the same prompt may produce noticeably different results on Telnyx vs Minimax. Try both.
- Ignoring the preview text — the text you provide for synthesis should match the voice’s intended use. Don’t use a cheerful script for a somber voice.
The Enhance button
The portal’s Enhance button uses AI to expand a short description into a detailed prompt.
This is a good starting point, but review the expanded prompt before generating — you may want to tweak specific dimensions.
Overview
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/concepts.md
What voice cloning does
Voice cloning captures a speaker’s vocal characteristics — timbre, cadence, accent, pronunciation — from a short audio sample and applies them to new speech synthesis. The clone is a representation of the voice, not a recording of it. The system learns patterns from your audio and encodes them into parameters that guide TTS. This means:- The cloned voice can say things the original speaker never said
- Clone quality is bounded by what the model can learn from your sample
- Poor recordings, background noise, or inconsistent delivery degrade the clone
What cloning doesn’t do
A clone is not a recording. It’s a statistical approximation of a voice — the model extracts patterns (formant frequencies, prosodic tendencies, spectral characteristics) and applies them during synthesis. This means:- Output passes through the TTS model, which has its own characteristics. A clone sounds like the speaker, but through the lens of the model.
- Quality has a ceiling set by your source audio. No amount of API parameters will fix a noisy or inconsistent recording.
- The clone may not handle speech styles far from the original sample well. A voice cloned from calm narration may sound different when asked to express strong emotion.
Two ways to create a clone
Both produce the same output: a voice clone with a voice ID you can use in production.
Recording best practices
- Match your recording to your use case. Don’t read a monotone script if you want an expressive clone. The AI replicates what it hears — including energy, emotion, and pacing.
- Speak clearly, avoid background noise. Use a decent microphone in a quiet space. Background noise gets cloned too. You don’t need a 100-300 USB condenser in a quiet room is sufficient.
- Avoid long pauses. The cloned voice will mimic pauses between sentences. Keep speech flowing naturally.
- Trim your recording. Speech from start to finish, no dead air at the beginning or end.
- Speak in the target language. If you want the clone to speak Spanish, record in Spanish.
- Keep it consistent. Same tone, accent, and energy throughout. Wide fluctuations confuse the model. The AI clones everything — including stutters, “uhms”, and inconsistencies.
- Aim for the right volume. Target -23 to -18 dB RMS with peaks no higher than -3 dB. Too quiet = noise floor issues. Too loud = clipping.
- Audio codec doesn’t matter much. MP3 at 128 kbps or above is fine. WAV is ideal but higher bitrate MP3 won’t noticeably hurt quality.
- Optimal duration by model:
- Qwen3TTS: 5–10 seconds. Auto-trims to 10s. More isn’t better.
- Ultra: Up to 10 seconds.
- Minimax: 1–2 minutes is the sweet spot. Longer recordings capture more vocal range, but beyond 3 minutes yields diminishing returns.
Quickstart
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/quickstart.md
Upload a file
Select Telnyx or Minimax using the provider toggle. In the Voice Design, click Upload Audio and choose your file or drag and drop it. Enter a name for the voice and select the gender. Click Clone Voice. The system processes the audio and creates a voice clone, typically in a few seconds.Record directly in the browser
Click Upload Audio, then select the Record tab. Select the language you’ll speak in. The system generates a reading script optimized for voice cloning. Click Start Recording and read the script clearly. It’s designed to capture the full range of phonemes. Listen to your recording. Re-record if needed, then click Clone Voice.Using the API
Clone with Telnyx (default)
Clone with Minimax
Supports longer audio (up to 5 minutes):Clone with Ultra model
Ultra clones use the higher-qualityUltra model. The request returns 202 Accepted — poll the clone’s status until it becomes active.
202 Accepted):
GET /v2/voice_clones until status is active.
SDK Examples
Clone with Telnyx (default)
Python
Node.js
Clone with Minimax
Minimax supports longer audio (up to 20MB) and uses thespeech-2.8-turbo model.
Python
Node.js
Clone with Ultra model
Ultra clones return202 Accepted and require polling until the status is active.
Python
Node.js
Parameters
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/parameters.md
Models
Set viamodel_id (body) on POST /v2/voice_clones/from_upload, or use provider (body) to select Minimax.
Audio Requirements
Body parameteraudio_file (multipart) on POST /v2/voice_clones/from_upload.
- Qwen3TTS: aim for 5–10 seconds. Longer isn’t better — auto-trims to 10s.
- Minimax: longer is better. 1–2 minutes of varied speech gives more vocal range.
The ref_text Parameter
Body parameter on POST /v2/voice_clones/from_upload. Optional.
A transcript of what’s being said in the audio. Improves clone quality by giving the model a text reference to align against.
Ultra Async Flow
Whenmodel_id is "Ultra", the API returns 202 Accepted instead of 201:
Responses
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/responses.md
Voice ID Format
Every clone response includes fields to construct the voice ID:{Provider}.{Model}.{provider_voice_id}
See Using Custom Voices for how to use these IDs across products.
Clone Status
Qwen3TTS and Minimax clones are always
active on creation.
Errors
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/clone-voice/errors.md
General Errors
These errors apply to all providers.Telnyx / Cartesia Errors
Minimax Errors
Using Custom Voices
Source: https://developers.telnyx.com/docs/voice/voice-design-lab/using-custom-voices.mdEvery voice clone gets a unique voice ID:
{Provider}.{Model}.{voice_id}
- Telnyx:
Telnyx.Qwen3TTS.33226e69-3abd-429b-b64a-86775c9b5850 - Minimax:
Minimax.speech-2.8-turbo.TB4ZMVKanThGeldiw8rLBEg21v4ifjUTRgLpkodJxpMYV
provider, provider_supported_models, and provider_voice_id fields.
AI Assistants
Select your custom voice in the assistant’s voice settings. Telnyx clones appear under Telnyx / Qwen3TTS, Minimax clones under Minimax.Call Control
Pass the voice ID in thevoice field of the speak command.
TTS WebSocket
Pass the voice ID as thevoice query parameter on the WebSocket URL.
See the TTS streaming guide for the full connection flow.
API Reference (Voice)
Audio
- Transcribe speech to text: Transcribe speech to text. This endpoint is consistent with the OpenAI Transcription API and may be used with the OpenAI JS or Python SDK.
Text To Speech Commands
- List available voices: Retrieve a list of available voices from one or all TTS providers. When
provideris specified, returns voices for that provider only. Otherwise, returns voic…
Voice Designs
- List voice designs: Returns a paginated list of voice designs belonging to the authenticated account.
- Create or add a version to a voice design: Creates a new voice design (version 1) when
voice_design_idis omitted. Whenvoice_design_idis provided, adds a new version to the existing design instead… - Get a voice design: Returns the latest version of a voice design, or a specific version when
?version=Nis provided. Theidparameter accepts either a UUID or the design name. - Rename a voice design: Updates the name of a voice design. All versions retain their other properties.
- Delete a voice design: Permanently deletes a voice design and all of its versions. This action cannot be undone.
- Download voice design audio sample: Downloads the WAV audio sample for the voice design. Returns the latest version’s sample by default, or a specific version when
?version=Nis provided. The `… - Delete a specific version of a voice design: Permanently deletes a specific version of a voice design. The version number must be a positive integer.
Voice Clones
- List voice clones: Returns a paginated list of voice clones belonging to the authenticated account.
- Create a voice clone from a voice design: Creates a new voice clone by capturing the voice identity of an existing voice design. The clone can then be used for text-to-speech synthesis.
- Create a voice clone from an audio file upload: Creates a new voice clone by uploading an audio file directly. Supported formats: WAV, MP3, FLAC, OGG, M4A. For best results, provide 5–10 seconds of clear spe…
- Update a voice clone: Updates the name, language, or gender of a voice clone.
- Delete a voice clone: Permanently deletes a voice clone. This action cannot be undone.
- Download voice clone audio sample: Downloads the WAV audio sample that was used to create the voice clone.