streaming_start is synchronous and might take longer to respond. answer and dial requests with stream_url can also be synchronous. We advise not using the 500 ms retry rule for these requests, and waiting for the response before retrying. See Command Retries.Requesting streaming using Dial Command
The requesting dial command can be extended in the following way to request streaming using WebSockets:stream_url- the destination address when the stream is going to be delivered.stream_track- specifies which track should be streamed, with possible options:inbound_track(default)-
outbound_track -
both_tracks
Requesting streaming using Answer Command
Using the same attributes as above, streaming can be requested while answering the call:Verify the WebSocket connection
Telnyx signs every media stream WebSocket connection, so you can confirm that a connection to yourstream_url comes from Telnyx. The signature uses your account’s Ed25519 key pair, the same one Telnyx uses to sign webhooks.
Each connection request includes two headers:
telnyx-signature-ed25519: the Base64-encoded Ed25519 signature.telnyx-timestamp: the Unix time, in seconds, when the request was signed.
{telnyx-timestamp}|{stream_url}, where stream_url is the exact URL you set when starting the stream, including any query string. To check it, verify the Base64-decoded telnyx-signature-ed25519 header over that exact string with your account’s Ed25519 public key, available in the Mission Control Portal.
Streaming process flow
When the WebSocket connection is established, the following event is being sent:streaming.started webhook is sent:
stream_id that identifies a particular stream:
The order of events is not guaranteed. Use
sequence_number or chunk to reorder them; see Frame numbering.Currently only one streaming/fork operation is supported per call. In case of requesting media forking the WebSocket stream will be stopped and replaced by a RTP connection.
Stopping a stream
Afterstreaming_stop or hangup, Telnyx normally delivers the media already queued for your WebSocket, then sends the stop frame and closes the connection with code 1000. If the connection is already failing, the stop frame may not arrive.
The streaming.stopped webhook can arrive before the WebSocket closes. Treat the stop frame or the WebSocket closing, whichever comes first, as the end of the stream.
Read from the WebSocket at least as fast as real time. If your application reads more slowly, queued media builds up and latency grows. A WebSocket that accepts no data for about 5 seconds is treated as disconnected; see Reconnects.
Bidirectional media streaming
Sending RTP stream
The RTP stream can be sent to the call using websocket. The functionality can be enabled by settingstream_bidirectional_mode to rtp. For dial command it should look as follows:
The number of bidirectional RTP streams per call is limited to 1.
RTP stream codec
There are the following codecs supported by bidirectional streaming:- PCMU, 8 kHz (default)
- PCMA, 8 kHz
- G722, 8 kHz
- OPUS, 8 kHz, 16 kHz
- AMR-WB, 8 kHz, 16 kHz
- L16, 16 kHz
When the audio is sent using a different encoding than on the call, it will be transcoded, which may cause a degradation in quality.
The L16 codec provides improved support for AI voice agent integrations by offering reduced latency and eliminating transcoding overhead when interfacing with many AI platforms that natively support linear PCM audio.
Sending media files
Media files can also be sent back to the call through the websocket. This is done similarly to the playback_start command when using a base64-encoded mp3 file in the payload. Send a packet to the websocket connection as follows:- Media payloads can only be submitted once per second.
- Media must be base64 encoded mp3.
Clear message
Sending a clear message will immediately stop the media playing on the stream and clear the media queue.Mark message
Mark messages can be used to keep track of media ending on the stream:- You can send a mark message to the stream after a media message.
- When the media immediately preceding the mark finishes you will receive the same mark back.
clear message will also result in all queued marks being sent back.
DTMF message
In case of DTMF events on the call, the following message will be sent over websocket:occurred_at field is a timestamp captured on the Telephony engine side. These are consumed as asynchronous events and in certain circumstances may arrive out of order. This field can be used to ensure proper order.
Error message
In case of any case of error during media streaming, an error frame in the following format is sent:Frame numbering
sequence_number and chunk are counted per stream and continue across reconnects.
sequence_numbercounts the frames Telnyx sends. Use it to order frames, not to detect loss: numbers can be skipped, and two frames can share a number.connectedanderrorframes don’t carry asequence_number.chunkcountsmediaframes, with one counter for both tracks. A gap inchunkmeans media was lost, but consecutive values don’t prove that none was: media dropped during a reconnect is never numbered.timestampis the RTP timestamp of the media packet, in the codec’s RTP clock units. It isn’t wall-clock time.
Reconnects
If the WebSocket connection closes unexpectedly, or accepts no data for about 5 seconds, Telnyx reconnects to yourstream_url. Reconnects are on by default. To turn them off, set enable_reconnect to false on streaming_start, answer, or dial.
After streaming has started, streams that process the call audio before sending it, such as translation streams, don’t reconnect: a disconnect ends them.
A reconnect:
- Uses the same
stream_url, including its query string, and the samestream_auth_token. Credentials in the URL must stay valid for the whole call. - Is signed the same way as the first connection, with a new
telnyx-timestamp. See Verify the WebSocket connection. - Starts with a new
connectedframe, then a newstartframe if streaming had already started.sequence_numberandchunkcontinue; they don’t reset.
- Your server closes the connection with code 1000, 1007, 1008, or 1011. The stream ends. If streaming had already started, you receive
streaming.stopped, notstreaming.failed. enable_reconnectisfalse. The stream ends at the first disconnect. If streaming had already started, you receivestreaming.stopped, notstreaming.failed.- Your server answers a reconnect attempt with a 4xx HTTP status. Telnyx stops retrying, and the stream fails.
- Media and events that arrive during a connection attempt are delivered late if that attempt succeeds. Those that arrive between attempts are dropped. Telnyx doesn’t replay them.
- No webhook is sent for a reconnect attempt or a successful reconnect.
- If the connection can’t be restored, you receive
streaming.failedwithfailure_reasonset todisconnected. The call continues.
Stream failures
When a stream fails, Telnyx sends thestreaming.failed webhook. Its failure_reason field says why:
New values may be added, so handle values you don’t recognize.