Stream call media over WebSocket
AsyncAPI specification for the Telnyx Media Streaming WebSocket connection.
Media Streaming forks call media and delivers it to a customer-provided WebSocket URL configured through Call Control using stream_url. Telnyx opens the WebSocket connection, sends lifecycle and media frames, and can receive bidirectional media playback commands from the connected application.
Connection Flow
- Start media streaming with a Call Control command and provide
stream_url. - Telnyx opens a WebSocket connection to the configured
stream_url. - Telnyx sends a
connectedframe, followed by astartframe that identifies the call and media format. - Telnyx sends
mediaframes with base64-encoded RTP payloads for the requested track or tracks. - If bidirectional streaming is enabled, the connected application can send
media,mark, andclearframes back to Telnyx. - Telnyx sends
dtmf,mark,error, andstopframes as applicable.
Bidirectional Media
Bidirectional streaming is enabled through Call Control by setting stream_bidirectional_mode to rtp for RTP payloads, or by sending base64-encoded MP3 media frames for media-file playback. RTP media chunks sent back to Telnyx can contain 20 milliseconds to 30 seconds of audio. MP3 media frames are queued and played in submission order, with a one-message-per-second rate limit.
Ordering
Media frame order is not guaranteed. Use media.chunk, media.timestamp, and sequence_number to order frames when required.
{
"event": "connected",
"version": "1.0.0"
}{
"event": "start",
"sequence_number": "1",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"start": {
"user_id": "3E6F995F-85F7-4705-9741-53B116D28237",
"call_control_id": "v2:T02llQxIyaRkhfRKxgAP8nY511EhFLizdvdUKJiSw8d6A9BborherQ",
"call_session_id": "ff55a038-6f5d-11ef-9692-02420aeffb1f",
"from": "+13122010094",
"to": "+13122123456",
"tags": [
"TAG1",
"TAG2"
],
"client_state": "aGF2ZSBhIG5pY2UgZGF5ID1d",
"custom_parameters": {
"customer_id": "customer_123"
},
"media_format": {
"encoding": "PCMU",
"sample_rate": 8000,
"channels": 1
}
}
}{
"event": "media",
"sequence_number": "4",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"media": {
"track": "inbound",
"chunk": "2",
"timestamp": "5",
"payload": "no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJev"
}
}{
"event": "stop",
"sequence_number": "5",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"stop": {
"user_id": "3E6F995F-85F7-4705-9741-53B116D28237",
"call_control_id": "v2:T02llQxIyaRkhfRKxgAP8nY511EhFLizdvdUKJiSw8d6A9BborherQ"
}
}{
"event": "mark",
"sequence_number": "5",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"mark": {
"name": "some_mark_name"
}
}{
"event": "dtmf",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"occurred_at": "2025-06-05T08:54:19.698408Z",
"sequence_number": "5",
"dtmf": {
"digit": "1"
}
}{
"event": "error",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"payload": {
"code": 100004,
"title": "invalid_media",
"detail": "Media provided was not base64 encoded"
}
}{
"event": "media",
"media": {
"payload": "eW91ciBiYXNlNjQgZW5jb2RlZCBhdWRpbyBwYXlsb2Fk"
}
}{
"event": "mark",
"mark": {
"name": "some_mark_name"
}
}{
"event": "clear"
}First frame sent by Telnyx after the WebSocket connection is established.
When a streaming auth token is configured, Telnyx includes it in the connected.x-telnyx-streaming-auth-token field and sends it as the x-telnyx-streaming-auth-token WebSocket header.
Frame sent before media begins. Identifies the stream, call metadata, custom parameters, and media format.
Telnyx-to-application media frame with a base64-encoded RTP payload without RTP headers.
Frame sent when Telnyx stops media streaming for the call.
Telnyx sends a mark callback when the media immediately preceding a submitted mark finishes playing. If no audio is playing or queued, the mark is returned immediately. A clear frame also returns queued marks.
Frame sent when Telnyx detects DTMF on the call.
Frame sent when an error occurs during media streaming.
Application-to-Telnyx media frame.
In RTP bidirectional mode, media.payload is a base64-encoded RTP payload. Supported RTP codecs are PCMU, PCMA, G722, OPUS, AMR-WB, and L16. For media-file playback, media.payload is a base64-encoded MP3 file.
Application-to-Telnyx mark frame used to track media completion. Send a mark after a media frame. Telnyx sends the same mark back when the immediately preceding media finishes playing, or immediately if no audio is queued.
Stops the current media playback and clears queued media. Queued marks are sent back by Telnyx.
Was this page helpful?
{
"event": "connected",
"version": "1.0.0"
}{
"event": "start",
"sequence_number": "1",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"start": {
"user_id": "3E6F995F-85F7-4705-9741-53B116D28237",
"call_control_id": "v2:T02llQxIyaRkhfRKxgAP8nY511EhFLizdvdUKJiSw8d6A9BborherQ",
"call_session_id": "ff55a038-6f5d-11ef-9692-02420aeffb1f",
"from": "+13122010094",
"to": "+13122123456",
"tags": [
"TAG1",
"TAG2"
],
"client_state": "aGF2ZSBhIG5pY2UgZGF5ID1d",
"custom_parameters": {
"customer_id": "customer_123"
},
"media_format": {
"encoding": "PCMU",
"sample_rate": 8000,
"channels": 1
}
}
}{
"event": "media",
"sequence_number": "4",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"media": {
"track": "inbound",
"chunk": "2",
"timestamp": "5",
"payload": "no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJev"
}
}{
"event": "stop",
"sequence_number": "5",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"stop": {
"user_id": "3E6F995F-85F7-4705-9741-53B116D28237",
"call_control_id": "v2:T02llQxIyaRkhfRKxgAP8nY511EhFLizdvdUKJiSw8d6A9BborherQ"
}
}{
"event": "mark",
"sequence_number": "5",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"mark": {
"name": "some_mark_name"
}
}{
"event": "dtmf",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"occurred_at": "2025-06-05T08:54:19.698408Z",
"sequence_number": "5",
"dtmf": {
"digit": "1"
}
}{
"event": "error",
"stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
"payload": {
"code": 100004,
"title": "invalid_media",
"detail": "Media provided was not base64 encoded"
}
}{
"event": "media",
"media": {
"payload": "eW91ciBiYXNlNjQgZW5jb2RlZCBhdWRpbyBwYXlsb2Fk"
}
}{
"event": "mark",
"mark": {
"name": "some_mark_name"
}
}{
"event": "clear"
}