Stream call media over WebSocket

AsyncAPI specification for the Telnyx Media Streaming WebSocket connection. Media Streaming forks call media and delivers it to a customer-provided WebSocket URL configured through Call Control using stream_url. Telnyx opens the WebSocket connection, sends lifecycle and media frames, and can receive bidirectional media playback commands from the connected application.

Connection Flow

Start media streaming with a Call Control command and provide stream_url.
Telnyx opens a WebSocket connection to the configured stream_url.
Telnyx sends a connected frame, followed by a start frame that identifies the call and media format.
Telnyx sends media frames with base64-encoded RTP payloads for the requested track or tracks.
If bidirectional streaming is enabled, the connected application can send media, mark, and clear frames back to Telnyx.
Telnyx sends dtmf, mark, error, and stop frames as applicable.

Bidirectional Media

Bidirectional streaming is enabled through Call Control by setting stream_bidirectional_mode to rtp for RTP payloads, or by sending base64-encoded MP3 media frames for media-file playback. RTP media chunks sent back to Telnyx can contain 20 milliseconds to 30 seconds of audio. MP3 media frames are queued and played in submission order, with a one-message-per-second rate limit.

Ordering

Media frame order is not guaranteed. Use media.chunk, media.timestamp, and sequence_number to order frames when required.

WSS

Messages

{
  "event": "start",
  "sequence_number": "1",
  "stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
  "start": {
    "user_id": "3E6F995F-85F7-4705-9741-53B116D28237",
    "call_control_id": "v2:T02llQxIyaRkhfRKxgAP8nY511EhFLizdvdUKJiSw8d6A9BborherQ",
    "call_session_id": "ff55a038-6f5d-11ef-9692-02420aeffb1f",
    "from": "+13122010094",
    "to": "+13122123456",
    "tags": [
      "TAG1",
      "TAG2"
    ],
    "client_state": "aGF2ZSBhIG5pY2UgZGF5ID1d",
    "custom_parameters": {
      "customer_id": "customer_123"
    },
    "media_format": {
      "encoding": "PCMU",
      "sample_rate": 8000,
      "channels": 1
    }
  }
}

{
  "event": "media",
  "sequence_number": "4",
  "stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
  "media": {
    "track": "inbound",
    "chunk": "2",
    "timestamp": "5",
    "payload": "no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJev"
  }
}

{
  "event": "stop",
  "sequence_number": "5",
  "stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
  "stop": {
    "user_id": "3E6F995F-85F7-4705-9741-53B116D28237",
    "call_control_id": "v2:T02llQxIyaRkhfRKxgAP8nY511EhFLizdvdUKJiSw8d6A9BborherQ"
  }
}

Connected Frame

type:object

First frame sent by Telnyx after the WebSocket connection is established. When a streaming auth token is configured, Telnyx includes it in the connected.x-telnyx-streaming-auth-token field and sends it as the x-telnyx-streaming-auth-token WebSocket header.

Start Frame

type:object

Frame sent before media begins. Identifies the stream, call metadata, custom parameters, and media format.

Telnyx Media Frame

type:object

Telnyx-to-application media frame with a base64-encoded RTP payload without RTP headers.

Stop Frame

type:object

Frame sent when Telnyx stops media streaming for the call.

Mark Callback Frame

Telnyx sends a mark callback when the media immediately preceding a submitted mark finishes playing. If no audio is playing or queued, the mark is returned immediately. A clear frame also returns queued marks.

DTMF Frame

type:object

Frame sent when Telnyx detects DTMF on the call.

Error Frame

type:object

Frame sent when an error occurs during media streaming.

Client Media Frame

type:object

Application-to-Telnyx media frame. In RTP bidirectional mode, media.payload is a base64-encoded RTP payload. Supported RTP codecs are PCMU, PCMA, G722, OPUS, AMR-WB, and L16. For media-file playback, media.payload is a base64-encoded MP3 file.

Mark Frame

type:object

Application-to-Telnyx mark frame used to track media completion. Send a mark after a media frame. Telnyx sends the same mark back when the immediately preceding media finishes playing, or immediately if no audio is queued.

Clear Frame

type:object

Stops the current media playback and clears queued media. Queued marks are sent back by Telnyx.

Call streaming stopped

Noise Suppression Start (BETA)

⌘I

Messages

{
  "event": "start",
  "sequence_number": "1",
  "stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
  "start": {
    "user_id": "3E6F995F-85F7-4705-9741-53B116D28237",
    "call_control_id": "v2:T02llQxIyaRkhfRKxgAP8nY511EhFLizdvdUKJiSw8d6A9BborherQ",
    "call_session_id": "ff55a038-6f5d-11ef-9692-02420aeffb1f",
    "from": "+13122010094",
    "to": "+13122123456",
    "tags": [
      "TAG1",
      "TAG2"
    ],
    "client_state": "aGF2ZSBhIG5pY2UgZGF5ID1d",
    "custom_parameters": {
      "customer_id": "customer_123"
    },
    "media_format": {
      "encoding": "PCMU",
      "sample_rate": 8000,
      "channels": 1
    }
  }
}

{
  "event": "media",
  "sequence_number": "4",
  "stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
  "media": {
    "track": "inbound",
    "chunk": "2",
    "timestamp": "5",
    "payload": "no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJev"
  }
}

{
  "event": "stop",
  "sequence_number": "5",
  "stream_id": "32DE0DEA-53CB-4B21-89A4-9E1819C043BC",
  "stop": {
    "user_id": "3E6F995F-85F7-4705-9741-53B116D28237",
    "call_control_id": "v2:T02llQxIyaRkhfRKxgAP8nY511EhFLizdvdUKJiSw8d6A9BborherQ"
  }
}

Documentation Index