Telnyx Video Overview

Use Telnyx for Real Time Communication embedded in your web applications.

Video is currently under development

The Video product and SDK enables you to:

Build mobile clients that embed real time communications
Generate on-demand access tokens for your clients
Add real time video communication to your web applications
Moderate your Video Room participants

HTTP ENDPOINTS

Video Room endpoints

https://api.telnyx.com/v2/rooms
https://api.telnyx.com/v2/rooms/{room_id}
https://api.telnyx.com/v2/rooms/{room_id}/sessions
https://api.telnyx.com/v2/rooms/{room_id}/actions/create_access_token
https://api.telnyx.com/v2/rooms/{room_id}/actions/refresh_access_token

Video Room Session endpoints

https://api.telnyx.com/v2/room_sessions
https://api.telnyx.com/v2/room_sessions/{room_session_id}
https://api.telnyx.com/v2/room_sessions/{room_session_id}/participants
https://api.telnyx.com/v2/room_sessions/{room_session_id}/actions/mute
https://api.telnyx.com/v2/room_sessions/{room_session_id}/actions/unmute
https://api.telnyx.com/v2/room_sessions/{room_session_id}/actions/kick

Video Room Participant endpoints

https://api.telnyx.com/v2/room_participants

Getting Started with Video

Requirements

To setup Video all you need is:

An API Key
A Client Token to join Video Rooms you will create

Configuration and Usage

Telnyx Video is enabled using Video Rooms. A Video Room represents communications session among multiple endpoints using one of Telnyx’s Programmable Video SDKs. Connected users (Participants) can share video and audio Tracks with the Room, and receive video and audio Tracks from other Participants in the Room. You can as many Rooms as you want. For example you could create a long lived Room such as "Daily Standup", or Rooms that you would delete after it's been used like "1-1 with X".

To create a Video Room you can use the REST API V2 documented here.

A Video Room can only be joined if the client owns a Client Join Token, you can create it using the REST API V2 documented here. The Client Join Token is short lived and you will be able to refresh it using the Refresh Token provided with it when you request for a Client Join Token.

Once you have a Video Room and an Client Join Token for it, you can then use our Video SDK on your client side to connect your client to the Room you just created.

Glossary

Term	Description
Video Room	Resource representing communications session among multiple endpoints using one of Telnyx’s Programmable Video SDKs
JWT	JSON Web Token. A standard method for representing claims (https://jwt.io/)
Client Join Token (JWT)	A JWT token which contains grants allowing in the Video Room usecase to join a Video Room
Refresh Token (JWT)	A JWT token which permits to obtain a new Client Token with same grants
API Key	Secret API Key generated via Portal and used to authenticate Telnyx API calls.
Video SDK	A library used to provide Video features to your application using Telnyx Video platform.

JavaScript Client SDK

The Telnyx Video Client SDK provides all the functionality you need to join and interact with a video room from a browser.

npm

Adding Telnyx to your JavaScript client application

Include the @telnyx/video npm module as a dependency:

npm install @telnyx/video --save

Then, import @telnyx/video in your application code.

// main.js
import { Room, createLocalParticipant } from '@telnyx/video';

Now you are ready to connect to a video room that you created. In order to connect to a video room you will require a client token that has the necessary grants to join the room.

const room = new Room(roomId, {
  clientToken: '<CLIENT_TOKEN_FOR_THE_ROOM>',
  localParticipant: createLocalParticipant({
    context: JSON.stringify({ name: 'Bob The Builder', id: 1 }), // send data that you can associate with this participant (for e.g., userId for this participant in your DB)
  }),
});

const stateCallback = (state) => {
  // the state object is immutable and can be easily integrated with most modern UI libraries like React, Vue etc.
};

room.on('state_changed', stateCallback);

room.connect().then(() => {
  console.log('You are connected to the room!');
});

Understanding the state of the video room

The state_changed event callback contains the state of the SDK at that point in time. This is an immutable object that you can use in most modern UI libraries like React and Vue.

The Typescript definition of the State is helpful in understanding the structure of this object.

type Status =
  | 'initialized'
  | 'connecting'
  | 'connected'
  | 'disconnecting'
  | 'disconnected';

interface State {
  status: Status;
  localParticipantId: Participant['id'];
  participants: {
    [id: string]: Participant;
  };
  streams: {
    [id: string]: Stream;
  };
}

Everytime the state of the SDK changes the state_changed callback is invoked with a new immutable state that represents the current state of the SDK. Since most modern UI libraries are able to compare the two immutable states and render only the components that changed it makes it easier to integrate the SDK with them rather than depending on multiple event callbacks.

For e.g., based on how we initialized it in the above code example the initial state of the SDK would result in an object give below. Note that the ID of the local participant is unique and are generated based on UUID v4 standard when you create the local participant.

{
  status: 'initialized',
  localParticipantId: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
  participants: {
    "8c3bacb5-2e90-4379-8ee8-d5446213fee9": {
      id: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
      context: "{\"name\":\"Bob The Builder\",\"id\":1}",
      streams: {},
    },
  },
  streams: {},
};

At this point there are no streams being published by the local participant. We will come to that shortly. The participant object follows the TypeScript interface given below.

interface Participant {
  id: string;
  context?: string;
  streams: {
    [key: string]: Stream['id'];
  };
}

When we called the room.connect() method in the example code the state of the SDK will change to the following ...

{
  status: 'connecting',
  localParticipantId: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
  participants: {
    "8c3bacb5-2e90-4379-8ee8-d5446213fee9": {
      id: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
      context: "{\"name\":\"Bob The Builder\",\"id\":1}",
      streams: {},
    },
  },
  streams: {},
}

This new state will be available to your application via the state_changed callback. Since state is immutable the only difference between the initial state and the new state is the status property. If you are using a modern UI library like React you can easily rerender the components to show that the application is connecting to the room.

When successfully connected the state changes to ...

{
  status: "connected",
  localParticipantId: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
  participants: {
    "8c3bacb5-2e90-4379-8ee8-d5446213fee9": {
      id: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
      context: "{\"name\":\"Bob The Builder\",\"id\":1}",
      streams: {},
    },
  },
  streams: {},
}

Now we are ready to start publishing streams on behalf of the local participant.

Publishing your local camera and mic stream

In order to publish a stream you need to define the constraints of the media. The simplest form of the constraints is ...

const constraints = { audio: true, video: true };

With these constraints the SDK will try to obtain both audio and video from the local participant. In order to publish the stream as the local participant you have to use the publish method available to you via the room object.

room.publish('self', {
  constraints: { audio: true, video: true },
});

The first argument to publish is a string that acts as the key that you can use to refer to this specific stream. You can use any valid string for this as long as you are consistent in your application. For e.g. here we're using 'self' for the camera/mic stream but you could use 'presentation' as the key for when you publish video of the screen. We will get to how you can publish your screen in a short while.

When you make the request to publish a stream, the browser will ask for the necessary permissions required to access the camera and mic. Once the permissions are acquired the SDK will configure and publish the stream to the room you are connected to.

At this point you will receive the new state to the state_changed event callback with the newly created stream.

{
  status: "connected",
  localParticipantId: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
  participants: {
    "8c3bacb5-2e90-4379-8ee8-d5446213fee9": {
      id: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
      context: "{\"name\":\"Bob The Builder\",\"id\":1}",
      streams: {
        self: "a87dd242-de78-4c19-a09c-23b336c9f25e",
      },
    },
  },
  streams: {
    "a87dd242-de78-4c19-a09c-23b336c9f25e": {
      id: "a87dd242-de78-4c19-a09c-23b336c9f25e",
      key: "self",

      // the constraits that was provided by you
      constraints: {
        audio: true,
        video: true,
      },
      bitrate: 256000, // also configurable when you publish a stream

      audioActive: false, // whether the audio track is being published
      videoActive: false, // whether the video track is being published
      source: MediaStream, // an instance of MediaStream that can be used to render video/audio
      audioTrack: undefined, // this will be an instance of MediaStreamTrack when the track is available
      videoTrack: undefined, // this will be an instance of MediaStreamTrack when the track is available

      isSpeaking: false, // whether the audio level of the track is high enough to consider the participant who owns this stream is speaking or not
      isRemote: false, // whether the stream originates from a remote source or not

      isPublishing: true, // whether the stream is being published
      isConfiguring: false, // whether the SDK is currently negotiating the WebRTC connection

      participantId: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
    },
  },
}

The stream object is the most complex object in the state of the SDK. However you don't have to worry about all of these properties at the moment.

As you can see the isPublishing property of the stream is true now. This means that the stream is being published, which involves creating the SDP (Session Description Protocol) and establishing the WebRTC connection. You can't publish another stream of the same key ("self" in our case) until this stream is published or removed (by unpublishing).

Once the SDK acquires the media tracks from the camera and mic of the local participant a new state will be returned using the state_changed callback. This state will contain the audio and video track from the local media devices.

The streams object at this point will look like this

  streams: {
    "a87dd242-de78-4c19-a09c-23b336c9f25e": {
      id: "a87dd242-de78-4c19-a09c-23b336c9f25e",
      key: "self",

      // the constraits that was provided by you
      constraints: {
        audio: true,
        video: true,
      },
      bitrate: 256000,

      audioActive: true,
      videoActive: true,
      source: MediaStream,
      audioTrack: MediaStreamTrack,
      videoTrack: MediaStreamTrack,

      isSpeaking: false,
      isRemote: false,

      isPublishing: true,
      isConfiguring: true,

      participantId: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
    },
  },

As you can see at this point you can use the source or the media tracks individually from audioTrack and videoTrack to show the media in your UI.

You can also see that the audioActive and videoActive flags are now true indicating the stream is publishing audio and video respectively.

Another property that was updated in the new state is the isConfiguring flag. You can ignore this in most use-cases as this indicates if the WebRTC connection is being negotiated.

Once the WebRTC connection is negotiated and the stream is successfully published the stream state would look like this

  streams: {
    "a87dd242-de78-4c19-a09c-23b336c9f25e": {
      id: "a87dd242-de78-4c19-a09c-23b336c9f25e",
      key: "self",

      // the constraints provided by you
      constraints: {
        audio: true,
        video: true,
      },
      bitrate: 256000,

      audioActive: true,
      videoActive: true,
      source: MediaStream,
      audioTrack: MediaStreamTrack,
      videoTrack: MediaStreamTrack,

      isSpeaking: false,
      isRemote: false,

      isPublishing: false,
      isConfiguring: false,

      participantId: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
    },
  },

At this point the stream will be available to be subscribed by other clients connected to the same room.

Knowing when a new participant joins the room

A video room can have multiple participants and you can get a list of all the participants connected to the room at the time by using the participants property in the state.

If the participant is not publishing any streams the streams property of that participant will be an empty object.

Let's imagine that another participant joins the same room from a different browser session with the following context

const room = new Room(roomId, {
  clientToken: '<CLIENT_TOKEN_FOR_THE_ROOM>',
  localParticipant: createLocalParticipant({
    context: JSON.stringify({ name: 'Oswald', id: 2 }), // send data that you can associate with this participant (for e.g., userId for this participant in your DB)
  }),
});

Since the room already contains a participant who is publishing a stream the state once this session is connected would look something like ...

{
  status: "connected",
  localParticipantId: "d42926f5-fe6c-48d5-b24b-7444048fa68e",
  participants: {
    "8c3bacb5-2e90-4379-8ee8-d5446213fee9": {
      id: "8c3bacb5-2e90-4379-8ee8-d5446213fee9",
      context: "{\"name\":\"Bob The Builder\",\"id\":1}",
      streams: {
        self: "a87dd242-de78-4c19-a09c-23b336c9f25e",
      },
    },
    "d42926f5-fe6c-48d5-b24b-7444048fa68e": {
      id: "d42926f5-fe6c-48d5-b24b-7444048fa68e",
      context: "{\"name\":\"Oswald\",\"id\":2}",
      streams: {},
    },
  },
  streams: {
    "a87dd242-de78-4c19-a09c-23b336c9f25e": {
      id: "a87dd242-de78-4c19-a09c-23b336c9f25e",
      key: "self",

      audioActive: true, // whether audio is being published by the remote stream
      videoActive: true, // whether video is bein published by the remote stream
      source: MediaStream,
      audioTrack: undefined, // will be an instance of MediaStreamTrack once subscribed
      videoTrack: undefined, // will be an instance of MediaStreamTrack once subscribed
      videoCodec: "vp8", // the codec used by the remote video
      audioCodec: "opus", // the codec used by the remote audio

      isRemote: true,
      isSpeaking: false, // whether the remote participant is speaking or not

      isConfiguring: false, // whether the WebRTC connection is being negotiated or not

      subscription: {
        status: "unsubscribed", // will change to 'subscribed' once the the client subscribes to this stream
      },

      participantId: "8c3bacb5-2e90-4379-8ee8-d5446213fee9", // the ID of the participant this stream belong to
    },
  },
}

As you can see there is already a participant in the participants object that maps to the remote particpant from the previous browser session. This remote participant also has a stream with the key "self".

The stream object for "self" has the structure very similar to the one we saw before but has a few new properties.

The ones that are particularly interesting are subscription, videoCodec, and audioCodec. The properties for video and audio codec can be used to decide if the browser has the capabilities to decode the video and audio tracks.

Then there is subscription.status, which is unpublished at the moment.

Subscribing to a remote stream

At this point we are ready to subscribe to the remote stream published by the participant. In order to do that we use the subscribe method from the room object.

room.subscribe('a87dd242-de78-4c19-a09c-23b336c9f25e');

This will start the WebRTC negotiation to receive the remote stream. Once successful the subscription.status will change to subscribed.

You can use the source property, which is an instance of MediaStream to render the media on the browser.

If the remote participant unpublishes this stream the SDK will automatically unsubscibe from the stream and the stream will disappear from the state.

Knowing when a participant is speaking

You can use the isSpeaking property in the stream object to know if the audio levels of that stream is high enough to be considered as speaking. Please note that this approach might change in the future.

Leaving the room

When the participant decides to leave the room you can use the disconnect method in the room instance to leave the room.

room.disconnect();

This will update the status property in the state to disconnecting and then to disconnected once the participant has left the room. On the remote side the participant and its associated streams will be unsubscribed (if already subscribed) and removed from the state.

Integrating Chat message feature in your application

You need to enable the message feature in the Telnyx Video SDK initialize using the property enableMessages: true

import {
  initialize,
  Room,
  State,
  Participant,
  Stream,
  Message,
} from '@telnyx/video';

roomRef.current = await initialize({
        ...,     
        enableMessages: true,
      });

Receiving messages in Chat

You will need to listen to message_received event from Telnyx Video SDK in your UI app.

This event is used to listen to all the incoming messages that were sent to the chat.

roomRef.current.on('message_received', (participantId, message, recipients, state) => {});

Sending messages in Chat

To send messages to the Chat you need to use the new method room.sendMessage in your UI app.

This method is a Promise.

Sending Global messages (all the participants will receive the message)

 await sendMessage(message: Message)

Sending Private messages (Only the selected recipients will receive the message)

 await sendMessage(message: Message, recipients?: Array<Participant['id']>)

Android Client SDK

The Telnyx Video Client SDK provides all the functionality you need to join and interact with a Telnyx Room from an Android application.

Project structure

SDK project: sdk module, containing all Telnyx SDK components as well as tests.
Demo application: app module, containing a sample demo application utilizing the sdk module.

Adding the SDK to your Android client application

Add Jitpack.io as a repository within your root level build file:

allprojects {
    repositories {
        ...
        maven { url 'https://jitpack.io' }
    }
}

Add the dependency within the app level build file:

dependencies {
    implementation 'com.github.team-telnyx:telnyx-video-android:<tag>'
}

Tag should be replaced with the release version.

Then, import the TelnyxVideo SDK into your application code at the top of the class:

import com.telnyx.video.sdk.*

The ‘*’ symbol will import the whole SDK which will then be available for use within that class.

NOTE: Remember to add and handle INTERNET, RECORD_AUDIO and ACCESS_NETWORK_STATE permissions in order to properly use the SDK

    <uses-permission android:name="android.permission.INTERNET"/>
    <uses-permission android:name="android.permission.CAMERA"/>
    <uses-permission android:name="android.permission.RECORD_AUDIO"/>
    <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>

Before connecting to a Room

Get an API Key

You'll need an API key which is associated with your Mission Control Portal account under API Keys. You can learn how to do that here.

An API key is your credential to access our API. It allows you to:

to authenicate to the Rest API
to manage your access tokens

Create a Room to join (if it doesn't exist)

In order to join a room you must create it, if it doesn't already exist. See our Room Rest API to create one.

There's also additional resources on other endpoints available to perform basic operations on a Room.

Generate an a client token to join a room

In order to join a room you must have a client token for that Room. The client token is short lived and you will be able to refresh it using the refresh token provided with it when you request a client token.

Please see the docs here to learn how to create a client token.

Now you are ready to connect to a video room that you previously created using the REST API.

Connect to Room

To connect, you'll need to provide a participantName that will identify your user in that room.

You'll also need to provide an instance of ExternalData that will contain a username of type String and an Integer id. You will also provide your Android application's context in context

room = Room(
    context = context,
    roomId = UUID.fromString(roomId),
    roomToken = tokenInfo.token,
    externalData = ExternalData(id = 1234, username = "Android Participant")
    enableMessages = false
)
...
...
room.connect()

Publish video/audio stream

To publish as video or audio stream, we will need an instance of PublishConfigHelper with the application context, camera direction, streamKey (unique for each stream published), and streamId (unique for each stream published)

//AUDIO
publishConfigHelper =
        PublishConfigHelper(
            context = requireContext(),
            direction = CameraDirection.FRONT,
            streamKey = SELF_STREAM_KEY // a key to identify this stream i.e: "self"
            streamId = SELF_STREAM_ID // RANDOM id to this stream i.e: "qlkj323kj423" 
        )
    publishConfigHelper.createAudioTrack(true, // isTrackEnabled?
                                        AUDIO_TRACK_KEY // i.e: "myMic", "002"
                                        )
...
...
room.addStream(publishConfigHelper) // This stream is new. addStream() is called.

New streams can be created via the PublishConfigHelper class for both audio and video. They can be created together or added later independently.

//VIDEO

//NOTE: in this case, video is published in the same stream as above
publishConfigHelper.setSurfaceView(selfSurfaceRenderer) //Provide SurfaceRenderer

publishConfigHelper.createVideoTrack(
            CapturerConstraints.WIDTH.value,// i.e: 1280
            CapturerConstraints.HEIGHT.value,// i.e: 720
            CapturerConstraints.FPS.value, //i.e: 30 (fps)
            true, // isTrackEnabled?
            VIDEO_TRACK_KEY //i.e: i.e "cameraFeed", "001"
        )
...
...
room.updateStream(publishConfigHelper) // Stream already created, therefore updateStream is called.

NOTE: since stream is already created, it is only necessary to add the video track to PublishConfigHelper, and "update" the stream.

Remove video/audio track

To remove a video or audio track, publishConfigHelper has to be modified in order to remove the unwanted track

//Considering publishConfigHelper is the same instance as above



publishConfigHelper?.let {
            it.stopCapture()   //In case of video, we "stop the capture", update the stream, and release the surface
            roomsViewModel.updateStream(it)
            selfSurface?.let { surface -> it.releaseSurfaceView(surface) }
        }
...

publishConfigHelper?.let {
            it.disposeAudio()    //In case of audio, we "dispose" audio and update the stream.
            roomsViewModel.updateStream(it)
        }

Remove stream

By removing the stream, we will remove all tracks added to it.

room.removeStream(SELF_STREAM_KEY) // a key to identify this stream i.e: "self"

Video/Audio Observables

The Telnyx Video SDK for android works with mutable live data, and all the information you need to build your UI is provided through observables that will contain the most up to date information of the state of the room, such as current participant list, talking events, stream information, participants added, participants leaving, etc

State Observable

    room.getStateObservable()
    // MutableLiveData<State>

This observable will provide the current state of a room at any given moment. We will receive a State object that will contain:

data class State(
    val action: String,
    val status: String,
    val participants: List<Participant>,
    val streams: HashMap<Long, Stream>,
    val publishers: List<Publisher>,
    val subscriptions: List<Subscription>
)

action -> val action: String`
is the cause of the latest change of State

enum class StateAction(val action: String) {
    INITIALIZING_ROOM("initializing room"),
    STATUS_CHANGED("status changed"),
    ADD_PARTICIPANT("add participant"),
    REMOVE_PARTICIPANT("remove participant"),
    ADD_STREAM("add stream"),
    REMOVE_STREAM("remove stream"),
    ADD_SUBSCRIPTION("add subscription"),
    UPDATE_SUBSCRIPTION("update subscription"),
    REMOVE_SUBSCRIPTION("remove subscription"),
    PUBLISH("publish"),
    UNPUBLISH("unpublish"),
    UPDATE_PUBLISHED_STREAM("update published stream"),
    AUDIO_ACTIVITY("audio activity")
}

status -> val status: String
is the status of the Room session, when that action happened

enum class Status(val status: String) {
    INITIALIZED("initialized"),
    CONNECTING("connecting"),
    CONNECTED("connected"),
    DISCONNECTING("disconnecting"),
    DISCONNECTED("disconnected")
}

participants -> val participants: List<Participant>

is the list of participants present in a room. A Participant is a UI representation in variables, for an attendee to the room session.

data class Participant(
    var id: Long,
    val participantId: String,
    var externalUsername: String? = null,
    val isSelf: Boolean,
    var streams: MutableList<ParticipantStream> = mutableListOf(),
    var isTalking: String?,
    var isAudioCensored: Boolean? = false,
    var audioBridgeId: Long? = null,
    var canReceiveMessages: Boolean = false
) : Serializable

data class ParticipantStream(
    val publishingId: Long? = null,
    var streamKey: String? = null,
    var audioEnabled: StreamStatus = StreamStatus.UNKNOWN,
    var videoEnabled: StreamStatus = StreamStatus.UNKNOWN,
    var audioTrack: AudioTrack? = null,
    var videoTrack: VideoTrack? = null
)

enum class StreamStatus(val request: String) {
    UNKNOWN("unknown"),
    ENABLED("enabled"),
    DISABLED("disabled")
}

streams -> val streams: HashMap<Long, Stream>
this hash map, contains a track of the currently available streams and the id of the publisher streaming them. A stream, will contain tracks for video, audio or both.

data class Stream(
    val id: String,
    val key: String,
    val participantId: String,
    val origin: String,
    var isAudioEnabled: Boolean? = null,
    var isVideoEnabled: Boolean? = null,
    var isAudioCensored: Boolean? = null,
    var isVideoCensored: Boolean? = null
)

publishers -> val publishers: List<Publisher>
This will track all publishers in the room. A Publisher is an instance of an attendee or participant sharing some stream content in the room. NOTE: a single Participant, sharing multiple streams can be also linked to multiple Publisher ids

data class Publisher(
    val audio_codec: String?,
    val video_codec: String?,
    val display: String, // see DisplayParameters.kt
    val id: Long,
    val talking: Boolean,
    val audio_moderated: Boolean? // aka Censored
)

data class DisplayParameters(
    val participantId: String,
    val telephonyEngineParticipant: Boolean? = null,
    val external: String? = null,
    val stream: StreamData? = null,
    val canReceiveMessages: Boolean? = null
)

subscriptions -> val subscriptions: List<Subscription>
Each time a publisher starts streaming information, we will have the option of subscribe/unsubscribe to/from it. This list will track all the subscriptions.

data class Subscription(
    var publisherId: Long,
    var status: SubscriptionStatus
)

enum class SubscriptionStatus(val status: String) {
    NEVER_REQUESTED("never_requested"),
    PENDING("pending"),
    STARTED("started"),
    PAUSED("paused")
}

Event observables

Some observables will have a MutableLiveData of Event. Event is a wrapper of LiveData, and provide the means to ensure we only handle an observable once, no matter how many times we're set to observe it. If the contents have already been handled, we won't get that content again, unless we peekContent() instead.

/**
 * Used as a wrapper for data that is exposed via a LiveData that represents an event.
 */
open class Event<out T>(private val content: T) {

    var hasBeenHandled = false
        private set // Allow external read but not write

    /**
     * Returns the content and prevents its use again.
     */
    fun getContentIfNotHandled(): T? {
        return if (hasBeenHandled) {
            null
        } else {
            hasBeenHandled = true
            content
        }
    }

    /**
     * Returns the content, even if it's already been handled.
     */
    fun peekContent(): T = content
}

Participants Observable

room.getParticipantsObservable()
// MutableLiveData<MutableList<Participant>>

This mutable list will be received as soon as we join a Room session, and contains a list of the participants already present in the room, including yourself. The SDK will keep this list updated but won't post the changes, so this observer won't be fired again. See Participant

By receiving this list we can initialize a recycler adapter in order to show participants:

roomsViewModel.getParticipants().observe(viewLifecycleOwner) { participants ->
	participants.let { participantList ->
		participantAdapter.setData(participantList)
		if (participantList.size > 0){
			selfParticipantId = participantList[0].participantId
			selfParticipantHandleId = participantList[0].id
		}
	}
}

Joined Room Observable

    room.getJoinedRoomObservable()
    // MutableLiveData<Event<RoomUI>>

This mutable will fire an event as soon as we have connected to a room, and we have retrieved an initial list of Participants

It is useful when we want to update UI as soon as we have joined the room sucessfully. NOTE: this will be different to Status-CONNECTED that will be issued when we have successfully joined our plugins to handle session audio. See Status

Joined Participant Observable

    room.getJoinedParticipant()
    //MutableLiveData<Event<Participant>>

We receive the participant that has joined the room after client has already joined. We can add the this reference to the list used in our adapter as:

roomsViewModel.getJoinedParticipant().observe(viewLifecycleOwner) { participantJoined ->
	participantJoined?.let { joinedParticipantEvent ->
		joinedParticipantEvent.getContentIfNotHandled()?.let {
			participantsAdapter.addParticipant(it)
		}
	}
}

Leaving participant id Observable

    room.getLeavingParticipantId()
    //MutableLiveData<Pair<Long, String>>

We receive the publisherId that has leaved the room, and a reason for its exit ("Left" or "Kicked").

roomsViewModel.getLeavingParticipantId()
    .observe(viewLifecycleOwner) { participantLeavingId ->
        participantLeavingId?.let { (id, reason) ->
            participantAdapter.removeParticipant(id)
            if (id == selfParticipantHandleId && reason == "kicked") {
                //It's ourselves, remove from the room.
                goBack(wasKicked = true)
                Toast.makeText(requireContext(), "You were kicked!", Toast.LENGTH_LONG).show()
            }
        }
    }
}

Connected to room Observable

    room.getConnectionStatus()
    //LiveData<Boolean>

Receive true when we have opened a webSocket and connected to the room

roomsViewModel.connectedToRoomObservable().observe(this.viewLifecycleOwner) {
    it?.let { isConnected ->
        if (isConnected) {
            buttonCreateRoom.isEnabled = true
        }
    }
}

Participant stream changed Observable

    room.getParticipantStreamChanged()
    //MutableLiveData<Event<Participant>>

We will receive here an event with the participant that has recently changed its video and/or audio stream status. Mutable list of participants will also be updated with this change, but here the specific individual is received.

Stream status

enum class StreamStatus(val request: String) {
    UNKNOWN("unknown"),
    ENABLED("enabled"),
    DISABLED("disabled")
}

These status apply for audio, video and shared screen:

UNKNOWN : initial status. We don't have information on whether this participant is sharing audio/video/screen

ENABLED : the participant is sharing audio/video/screen and we can subscribe to a stream for it like:

DISABLED : the participant is not sharing audio/video/screen. If we were subscribed to that participant audio/video/screen we can unsubscribe from it.

In order to subscribe to a video stream, there are 3 actions that needs to be performed:

StreamStatus.ENABLED -> {        
    // This notifies the WebRTC connection we're ready to receive stream information
    participantTileListener.subscribeTileToStream(model.participantId, "self")
    
    itemView.participant_tile_surface.visibility = View.VISIBLE
    itemView.participant_tile_place_holder.visibility = View.GONE
    
    // This ensures surfaces are initialized in an EglContext provided inside WebRTC connection
    participantTileListener.notifyTileSurfaceId(
        itemView.participant_tile_surface,
        model.participantId,
        "self"
    )
    
    model.streams.find { it.streamKey == "self" }?.videoTrack?.let {
        if (viewHolderMap[holder] != it) {  // NOTE: keep a map of surfaces to release
            
            // Updates only if previous register differs from what we need
            viewHolderMap[holder]?.removeSink(holder.itemView.participant_tile_surface)
            holder.itemView.participant_tile_surface.release()
            viewHolderMap[holder] = it

            it.addSink(itemView.participant_tile_surface)
            it.setEnabled(true)
        }
    }
}

1 Provide the SDK with the same instance of SurfaceViewRenderer you want to init:

    room.setParticipantSurface(
        participantId: String,
        surface: SurfaceViewRenderer,
        streamKey: String //Stream key to indentify the webrtc connection to init this surface
    )

This will use the proper WebRTC Connection to provide an EglContext for the surface to be init

2 Use method addSink() to add a SurfaceViewRenderer instance to the videoTrack provided for a Participant, and set that track to enable videoTrack?.setEnabled(true)

3 Subscribe to the stream a participant is providing

    
    participantTileListener.subscribeTileToStream(model.participantId, "self")

    // Eventually calls:
 
    room.addSubscription(
        participantId: String,
        streamKey: String,
        streamConfig: StreamConfig,
    )

participantId is the participant id that uniquely identifies a single participant in the room

streamKey i.e: "SharingSubscriptions" "CameraSubscriptions"

streamConfig whether we want to subscribe to audio/video or both. By default we will attempt both.

Remove subscription to a *video stream

To remove a subscription we need to issue

    room.removeSubscription(participantId: String, streamKey: String)

A good practice when handling surfaces is to make sure you remove this surface properly from the rendering context before eliminating or removing the surface from the UI:

    // Here, order is important
    videoTrack.removeSink(surfaceViewRenderer)
    surfaceViewRenderer.release()

Participant Talking Observable

    room.getParticipantTalking()
    MutableLiveData<Pair<Participant, String?>>

We will receive the participant that has updated its talking status, and the stream key. This information is also modified in the Participants list

data class Participant(
...
    var isTalking: String?, // Can either be "talking" or "stopped-talking"
...
) : Serializable

Stats

We provide the method getWebRTCStatsForStream() to retrieve WebRTC stats This request brings stats one time only, so if you want to keep receiving stats, you will need to use some sort of runnable or coroutine to recursively request for them such as:

mStatsjob = CoroutineScope(Dispatchers.Default).launch {
        while (isActive) {
            room.getWebRTCStatsForStream(participantId, streamKey, callback)
            delay(2000)
    }

We have to provide participantId, streamKey that is the key that identifies the stream we need the stats from, and callback that is an RTCStatsCollectorCallback we provide to the WebRTC peer connection in order to retrieve the stats.

In Kotlin, method call will look like this

room.getWebRTCStatsForStream(participantId, streamKey) { stats ->
    ...
    ...
}

We can later parse the information retrieved to obtain the specific information we go after. In our sample app, you will see the models we use for audio and video, both local and remote

Remote Video stats

WEBRTC's RTCStatsReport can be parsed and mapped to RemoteVideoStreamStats

data class RemoteVideoStreamStats(
    val bytesReceived: Int,
    val frameHeight: Int,
    val frameWidth: Int,
    val framesDecoded: Int,
    val framesDropped: Int,
    val framesPerSecond: Double,
    val framesReceived: Int,
    val packetsLost: Int,
    val packetsReceived: Int,
    val totalInterFrameDelay: Double
) : StreamStats()

stats.statsMap.values.filter { it.type == "inbound-rtp" }
    .findLast { it.toString().contains("mediaType: \"video\"") }
    ?.let { rtcVideoStats ->
        val videoStreamStats =
            gson.fromJson(
                rtcVideoStats.toString(),
                RemoteVideoStreamStats::class.java
            )
        videoStreamStats?.let {
            Timber.tag("RoomFragment")
                .d("ParticipantID: $participantId video STATS: $it")
            // Proceed to use stats
        }
    }

Local Video stats

WEBRTC's RTCStatsReport can be parsed and mapped to LocalVideoStreamStats

data class LocalVideoStreamStats(
    val bytesSent: Int,
    val codecId: String,
    val frameHeight: Int,
    val frameWidth: Int,
    val framesEncoded: Int,
    val framesPerSecond: Double,
    val framesSent: Int,
    val headerBytesSent: Int,
    val nackCount: Int,
    val packetsSent: Int,
) : StreamStats()

stats.statsMap.values.filter { it.type == "outbound-rtp" }
    .findLast { it.toString().contains("mediaType: \"video\"") }
    ?.let { rtcVideoStats ->
        val videoStreamStats =
            gson.fromJson(
                rtcVideoStats.toString(),
                LocalVideoStreamStats::class.java
            )
        videoStreamStats?.let {
            Timber.tag("RoomFragment")
                .d("SelfParticipant video STATS: $it")
            // Proceed to use stats
        }
    }

Remote audio stats

WEBRTC's RTCStatsReport can be parsed and mapped to RemoteAudioStreamStats

data class RemoteAudioStreamStats(
    val audioLevel: Double,
    val bytesReceived: Int,
    val codecId: String,
    val headerBytesReceived: Int,
    val jitter: Double,
    val packetsLost: Int,
    val packetsReceived: Int,
    val totalAudioEnergy: Double,
    val totalSamplesDuration: Double,
    val totalSamplesReceived: Int,
) : StreamStats()

stats.statsMap.values.filter { it.type == "inbound-rtp" }
    .findLast { it.toString().contains("mediaType: \"audio\"") }
    ?.let { rtcStats ->
        val audioStreamStats =
            gson.fromJson(
                rtcStats.toString(),
                RemoteAudioStreamStats::class.java
            )
        audioStreamStats?.let {
            // Proceed to use stats
        }
    }

Local audio stats

WEBRTC's RTCStatsReport can be parsed and mapped to LocalAudioStreamStats

data class LocalAudioStreamStats(
    val bytesSent: Int,
    val codecId: String,
    val headerBytesSent: Int,
    val packetsSent: Int,
    val retransmittedBytesSent: Int,
    val retransmittedPacketsSent: Int,
) : StreamStats()

stats.statsMap.values.filter { it.type == "outbound-rtp" }
    .findLast { it.toString().contains("mediaType: \"audio\"") }
    ?.let { rtcStats ->
        val audioStreamStats =
            gson.fromJson(
                rtcStats.toString(),
                LocalAudioStreamStats::class.java
            )
        audioStreamStats?.let {
            // Proceed to use stats
        }
    }

iOS Client SDK

The Telnyx Video iOS SDK provides the functionality you need to join and interact with a video room from an iOS application.

If you prefer to jump right in

Have a look at our demo app: Telnyx Meet.

An overview of the Video API

These are important concepts to understand.

A Room represents a real time audio/video/screen share session with other people or participants. It is fundamental to building a video application.
A Participant represents a person inside a Room. Each Room has one Local Participant and one or more Remote Participants.
Room State tracks the state of the room as it changes making it extremely easy to understand what's happened to a Room.
- Room State could change due to a Local Participant has started publishing a stream or because a Remote Participantleft.
A Stream represents the audio/video media streams that are shared by Participants in a Room
- A Stream is indentified by it's participantId and streamKey
A Participant can have one or more Stream's associated with it.
A Subscription is used to subscribe to a Stream belonging to a Remote Participant

API of a Room

This should give you high level overview of the Room API and it's functionality.

    func connect(statusChanged: @escaping (_ status: RoomStatus) -> Void)

    func disconnect(completion: @escaping () -> Void)

    func updateClientToken(clientToken: String, completion: () -> Void)
    
    func addStream(key: StreamKey, audio: RTCAudioTrack?, video: RTCVideoTrack?, completion: 
    @escaping OnSuccess, onFailed: @escaping OnFailed)

    func updateStream(key: StreamKey, audio: RTCAudioTrack?, video: RTCVideoTrack?, completion: @escaping OnSuccess, onFailed: @escaping OnFailed)

    func removeStream(key: StreamKey, completion: @escaping OnSuccess, onFailed: @escaping OnFailed)

    func addSubscription(participantId: ParticipantId, key: StreamKey, audio: Bool, video: Bool, completion: @escaping OnSuccess, onFailed: @escaping OnFailed)

    func pauseSubscription(participantId: ParticipantId, key: StreamKey, completion: @escaping OnSuccess, onFailed: @escaping OnFailed)

    func resumeSubscription(participantId: ParticipantId, key: StreamKey, completion: @escaping OnSuccess, onFailed: @escaping OnFailed)

    func updateSubscription(participantId: ParticipantId, key: StreamKey, audio: Bool, video: Bool, completion: @escaping OnSuccess, onFailed: @escaping OnFailed)

    func removeSubscription(participantId: ParticipantId, key: StreamKey, completion: @escaping OnSuccess, onFailed: @escaping OnFailed)

    func getWebRTCStatsForStream(participantId: ParticipantId, streamKey: StreamKey, completion: @escaping (_ stats: [String: [String: Any]]) -> Void)
    
    /// Helpers methods
    func getState() -> State
    func getLocalParticipant() throws -> Participant
    func getLocalStreams() throws -> [StreamKey: Stream]
    func getParticipantStream(participantId: ParticipantId, key: StreamKey) -> Stream?
    func getParticipantStreams(participantId: ParticipantId) throws -> [StreamKey: Stream]

Events that are triggered in a Room

Here's a list of events that will fire as you make API calls.

  /// Triggered each time the state is updated.
  var onStateChanged: ((_ state: State) -> Void)?

  /// Triggered when connected to a room.
  var onConnected: (() -> Void)?

  /// Triggered when disconnects from room / leaves room.
  var onDisconnected: (() -> Void)?

  /// Triggered when a remote participant joins the room.
  var onParticipantJoined: ((_ participantId: ParticipantId, _ participant: Participant) -> Void)? 

  /// Triggered when a remote participant leaves the room.
  var onParticipantLeft: ((_ participantId: ParticipantId) -> Void)?

  /// Triggered after successfully registering a stream.
  var onStreamPublished: ((_ participantId: ParticipantId, _ streamKey: StreamKey) -> Void)?

  /// Triggered after successfully unregistering a stream.
  var onStreamUnpublished: ((_ participantId: ParticipantId, _ streamKey: StreamKey) -> Void)?

  /// Triggered when a local stream or a remote stream track has been enabled.
  /// Notifies consumers about remote stream tracks being enabled. For example: when audio is unmuted or video has started on a remote stream.
  var onTrackEnabled: ((_ participantId: ParticipantId, _ streamKey: StreamKey, _ kind: String) -> Void)?

  /// The oposite of` onTrackEnabled`. Triggers when a local stream or a remote stream track has been disabled.
  /// Notifies consumers about remote stream tracks being disabled. For example: when audio is muted or video has stopped on a remote stream.
  var onTrackDisabled: ((_ participantId: ParticipantId, _ streamKey: StreamKey, _ kind: String) -> Void)?

  /// Triggered when subscribed to a remote participant's stream.
  var onSubscriptionStarted: ((_ participantId: ParticipantId, _ streamKey: StreamKey) -> Void)?

  /// Triggered when an ongoing subscription is paused.
  var onSubscriptionPaused: ((_ participantId: ParticipantId, _ streamKey: StreamKey) -> Void)?

  /// Triggered when a paused subsription is resumed.
  var onSubscriptionResumed: ((_ participantId: ParticipantId, _ streamKey: StreamKey) -> Void)?

  /// Triggered when the subscription is reconfigured.
  var onSubscriptionReconfigured: ((_ participantId: ParticipantId, _ streamKey: StreamKey) -> Void)?

  /// Triggered when subscription is ended for a remote participant's stream.
  /// The subscription can be ended by calling `removeSubscription(ParticipantId,StreamKey)` or when the remote participant leaves.
  var onSubscriptionEnded: ((_ participantId: ParticipantId, _ streamKey: StreamKey) -> Void)?

  /// onError
  /// Triggered when there's an error processing incoming events from the server.
  var onError: ((_ error: SdkError) -> Void)?

Understanding the state of the Room

Everything in the SDK centers around the Room object. When the state of the Room changes (e.g. a new participant joins or a remote participant starts publishing a stream) the onStateChanged event is triggered.

The event is invoked with a state parameter which contains the current of the state of the Room.

 /// Triggered each time the state is updated.
 var onStateChanged: ((_ state: State) -> Void)?

Before getting started

Install the SDK

Currently, the Telnyx iOS Video SDK can be installed using CocoaPods. For instructions on that check out our releases repo for iOS

Get an API Key

You'll need an API key which is associated with your Mission Control Portal account under API Keys. You can learn how to do that here.

An API key is your credential to access our API. It allows you to:

to authenicate to the Rest API
to manage your access tokens

Create a Room to join (if it doesn't exist)

In order to join a room you must create it, if it doesn't already exist. See our Room Rest API to create one.

There's also additional resources on other endpoints available to perform basic operations on a Room.

Generate an a client token to join a room

Please see the docs here to learn how to create a client token.

Code Examples

Enough already let's get to the code. Here are some code examples to get your wet feet on how to start building something with the iOS video SDK.

Participating in a Room

Connect to a room

First, you'll need to create a Room instance and then connect to it. Once you're connected to a room, you can start sharing audio/video streams with other participant in the rooms.

Important Note:
This simply creates an instance of a Room in code it does not use the Rooms Rest API to create a room, mentioned above in "Create a Room to join"*

// Create an instance of a Room

Room.createRoom(          
            id: "92f83cf907b6426197ca6ccc83f3cba3",
            clientToken: accessToken,
            context: ["userid": 12345, "username": "jane doe"])
{ room in
  // Once a room is created we can connect to it
  room.connect { status in
    
  }
}

Once the room is connected you've joined the room as it's local participant. You can see this more clearly by, after connecting, get the local participant.

Important Note:
Room only has one Local Participant but can have multiple Remote Participants.

Room.createRoom(
            id: "92f83cf907b6426197ca6ccc83f3cba3",
            clientToken: accessToken,
            context: ["username": "jane doe"])
{ room in        
    room.connect { status in
        let localParticipant = room.getLocalParticipant()
    }
}

What is Room context?

Context is any details you want to include about the LocalParticipant of the Room.

For instance, let's say you want to use context to identify a Participant with fields from an external system. You could pass a userId and username as context, like the code snippet above.

These details will be available to all RemoteParticipants in the Room, when they are notified about your presence in the Room.

Working with local media

Publishing audio and/or video from your camera or microphone works by using MediaDevices. MediaDevices is a helper class that we provide for you to make it easy to grab local media from your device.

let stream = MediaDevices.shared().getUserMedia(audio:true, video:true)

// If you want to run your app in a simulator provide a video file name to MediaDevices and it wil be used as the source for the cameraTrack. The video needs to be added to your Main.bundle, for things to work properly. 

let cameraTrack = stream.videoTracks.first
let microphoneTrack = stream.audioTracks.first

Setting the quality of the local video

You can set the camera resolution and fps using MediaDevices.

#if !targetEnvironment(simulator)
guard let camera = RTCCameraVideoCapturer.captureDevices().first(where: {
    $0.position == MediaDevices.shared().cameraPosition }) else {
        return
    }
// Choose a suitable resolution/capture format
guard let captureFormat = RTCCameraVideoCapturer.supportedFormats(for: camera).sorted { (f1, f2) -> Bool in
    let width1 = CMVideoFormatDescriptionGetDimensions(f1.formatDescription).width
    let width2 = CMVideoFormatDescriptionGetDimensions(f2.formatDescription).width
    return width1 < width2
}.first else {
    return
}
// Choose a suitable fps
let fps = captureFormat.videoSupportedFrameRateRanges.sorted { return $0.maxFrameRate < $1.maxFrameRate }.first!
// Set the resolution and fps to `Mediadevices`.
MediaDevices.shared().set(format: captureFormat, fps: Int(fps.maxFrameRate))
#endif

Note: If you choose highest resolution and fps, the local video stream will lag if you have poor internet / bandwith

Publishing a stream

Once you have tracks from say, a local media device like video from a camera and/our audio from your micrphone you can use those to create a Stream and publish it in the Room.

room.connect { status in        
  let cameraTrack: RTCVideoTrack
  let microphoneTrack: RTCAudioTrack

  // The onStreamPublished will trigger once the stream has started publishing in the room
  room.onStreamPublished = {
          participantId, streamKey in
          
  }
  
  room.addStream(
      key: "camera/mic",
      audio: microphoneTrack,
      video: cameraTrack){

  }
}

Unpublishing a stream

If you can longer want to continue publishing a stream you can unpublish it. Naturally the stream you want to unpublish must be added already.

room.connect { status in
  // onStreamUnpublished will trigger once the stream has been unpublished
  room.onStreamUnpublished = {
      participantId, streamKey in
      
  }
      
  room.removeStream(key: "camera/mic") {
      
  }
}

Working with Remote Participants and Streams

A remote participant who joins or leaves the room

When a remote participant joins a a room you will be notified with the Room.onParticipantJoined event. And similiarly with Room.onParticipantLeaving when a remote participant leaves.

You can use these events to keep track of participants in the room.

room.connect {
  room.onParticipantJoined = {
    participantId in
    // This event will trigger when a remote participant joins the room 
  }
}

Remote participants already in the room

When you connect to a Room there may already be remote participants in the Room. To understand who is in the Room after you connect use the onParticipantJoined event.

room.connect {
  room.onParticipantJoined = {
    participantId in
    // The event triggers for remote participants who are already in the room, just like it does for a new remote participant that joins the room.
  }
}

Display a remote participant's media

In order to understand how to display a remote participants' media let's review on subscriptions work in the API.

It might be helpful to review

the Video API overview.

to get a better understanding of how a Room is modeled, especially Stream and Subscription.

Major 🔑 about Subscriptions
In order to display media from a remote stream you need to subscribe to it.

It's important to understand that your Room doesn't automatically subscribe to a remote stream being published. It's your choice to decide whether to subscribe to a given stream.

Subscribing to a stream

So let's say the app your building has two users - let's them call them Alice and Bob.

First, Alice joins the Room and starts publishing a stream with audio from her microhone and video from her camera like this:
NOTE: The app on Alice's device runs the following code...

room.connect{
  status in
  // Let's assume that we have the tracks for Alice's camera and microphone already

  // Alice starts publishing a stream in the room
  room.addStream(
      key: "self",
      audio: microphoneTrack,
      video: cameraTrack){
  }
}

Bob wants to get Alice's stream so he can display it. In order to do so he needs to subscribe to Alice's stream.
NOTE: The app on Bob's device runs the following code...

room.connect { status in            
  // onStreamPublished event is triggered notifying him that Alice's stream is being published
  room.onStreamPublished = {
          participantId, streamKey in
          
          // Bob subscribes to Alice's stream
          room.addSubscription(
            participantId: participantId, 
            key: streamKey, 
            audio: true, 
            video: true
          )

  }

  // onSubscriptionStarted triggers when the subscription to Alice's stream has started
  room.onSubscritionStarted = {
    participantId, streamKey in    
    // Bob needs to fetch the stream so he can display it
    let aliceStream = room.getParticipantStream(participantId: participantId, key: streamKey)

    // Alice's stream has a key of 'self' which has the audio track from her microphone and a video track from her device's camera. Bob can use these tracks and display them as Alice in his app.
    let aliceCameraTrack = aliceStream.videoTrack
    let aliceMicrophoneTrack = aliceStream.audioTrack
  }
}

Handling remote streams that are already publishing in the Room

After you connect to a Room there may be remote participants already in the Room who are publishing streams in the Room.

To deal with that use the onStreamPublished event:

room.connect { status in            
  // After connecting to a room the onStreamPublished event will trigger for remote stream
  // that are already being published in the room
  // The onStreamPublished event will trigger 
  room.onStreamPublished = {
          participantId, streamKey in

          
  }
}

Disconnecting from a Room

To disconect from a room do:

room.disconnect {
  // after the room disconnect that it's status is .disconnected
}

When you disconnect from a Room all Remote Participant's will be notified that you've left the Room because the Room.onParticipantLeft event will fire on their Room instance.

Starting a Video Composition

Create or update a room

One can create a room with recording enabled and a webhook_event_url using the following request:

curl -X "POST" "https://api.telnyx.com/v2/rooms" \
     -H 'Authorization: Bearer <my-telnyx-api-key>' \
     -H 'Content-Type: application/json; charset=utf-8' \
     -d $'{
  "enable_recording": true,
  "webhook_event_url": "<my-webhook-url>",
  "unique_name": "my-unique-room-name"
}'

If successful, you'll get the room id in the response.

It's also possible to enable recording for an existing room with:

curl -X "PATCH" "https://api.telnyx.com/v2/rooms/<my-room-id>" \
     -H 'Authorization: Bearer  <my-telnyx-api-key>' \
     -H 'Content-Type: application/json; charset=utf-8' \
     -d $'{
  "enable_recording": true
}'

Please note any currently active session in the room will NOT have recording enabled, and it only enables recording for new sessions created after.

Also if the room doesn't already have webhook_event_url set, one can update it and add one:

curl -X "PATCH" "https://api.telnyx.com/v2/rooms/<my-room-id>" \
     -H 'Authorization: Bearer  <my-telnyx-api-key>' \
     -H 'Content-Type: application/json; charset=utf-8' \
     -d $'{
  "webhook_event_url": "<my-webhook-url>"
}'

Get the session id

If webhook_event_url is set for the room and there is no active session for the room, when the first participant joins, a new session will be created and you should receive a session started webhook event, which looks like this:

{
  "data": {
    "event_type": "video.room.session.started",
    "id": "529eac66-5992-46fc-9758-2a80765bb714",
    "occurred_at": "2022-03-01T02:38:52.038150Z",
    "payload": {
      "room_id": "fbd8f77f-5d8c-4e24-90a9-ca31ca97a0d4",
      "session_id": "cedd5466-3189-4aad-8b8b-52bd5094a906"
    },
    "record_type": "event"
  },
  "meta": {
    "attempt": 1,
    "delivered_to": "https://webhook.site/ae2228cb-b414-4428-ac5a-950a7e6603bc"
  }
}

And that would be a convenient way to get the session id.

If webhook_event_url is not set for the room, you can get the session id while the session is active with:

curl "https://api.telnyx.com/v2/rooms/<my-room-id>" \
     -H 'Authorization: Bearer <my-telnyx-api-key>'

There is a active_session_id field in the response, which is the id of the currently active session.

Get recording ids

If webhook_event_url is set for the room, when participants join and start video, you would receive recording started webhook events for video recordings:

{
  "data": {
    "event_type": "video.room.recording.started",
    "id": "3e2f1093-b0e2-4676-94f1-f388146bdd2f",
    "occurred_at": "2022-03-01 05:49:19.679223Z",
    "payload": {
      "participant_id": "77082c8c-9fff-4ba9-a7e9-46be410eb2c6",
      "recording_id": "3e8512e2-65d2-43ce-86d8-fb0016cf7a01",
      "room_id": "2201b930-bc70-450a-91e7-795d95303662",
      "session_id": "e4aa5752-208d-4e3e-bc10-cea1b60eee04",
      "type": "video"
    },
    "record_type": "event"
  },
  "meta": {
    "attempt": 1,
    "delivered_to": "https://webhook.site/ae2228cb-b414-4428-ac5a-950a7e6603bc"
  }
}

You can collect the relevant recording ids, which will be used as video_sources when creating a video composition later.

Note: if a participant unmute mic, you'd also receive a recording started event for the particpant's audio recording. But at this stage we are using mixed audio recording in composition, so you can ignore those events for the purpose of video composition.

If webhook_event_url is not set for the room, you can get the recording ids with:

curl "https://api.telnyx.com/v2/room_recordings/?filter\[session_id\]=<my-session-id>" \
     -H 'Authorization: Bearer <my-telnyx-api-key>'

You'll get a list of recordings in the response. Please make sure that all of video recordings have status as completed before creating a video composition for the session.

Create a video composition

A video composition can be created with the following request:

curl -X "POST" "https://api.telnyx.com/v2/room_compositions" \
     -H 'Authorization: Bearer <my-telnyx-api-key>' \
     -H 'Content-Type: application/json; charset=utf-8' \
     -d $'{
  "session_id": "<my-session-id>",
  "video_layout": {
    "presentation": {
      "z_pos": "1",
      "video_sources": [
        <recording id for sceensharing>
      ]
    },
    "participants": {
      "z_pos": "2",
      "x_pos": "10",
      "y_pos": "530",
      "width": "1260",
      "height": "160",
      "max_rows": "2",
      "video_sources": <list of ids for participant's video recordings>
    }
  },
  "webhook_event_url": "<my-webhook-url>",
  "resolution": "1280x720"
}'

Note: you can create a video composition before the session ends, but processing of the composition won't start until the session has ended and all the recordings are available.

If webhook_event_url is set for the composition, you would receive a composition completed event with Amazon S3 download url for the resulting video:

{
  "data": {
    "event_type": "video.room.composition.completed",
    "id": "f7543d82-b7c2-49ea-b44c-0d1fb90aa8cb",
    "occurred_at": "2022-03-02 02:32:29.713160Z",
    "payload": {
      "composition_id": "2f9521f7-2cb5-4eec-b9f1-09e448d9b234",
      "download_url": "[REDACTED AWS S3 URL]",
      "duration_secs": 154,
      "format": "mp4",
      "resolution": "1280x720",
      "room_id": "2201b930-bc70-450a-91e7-795d95303662",
      "session_id": "e4aa5752-208d-4e3e-bc10-cea1b60eee04",
      "size_mb": 9.5
    },
    "record_type": "event"
  },
  "meta": {
    "attempt": 1,
    "delivered_to": "https://webhook.site/ae2228cb-b414-4428-ac5a-950a7e6603bc"
  }
}

Note: webhook_event_url for a composition is independent of the webhook_event_url for a room. If you want to receive composition related webhook events, you'll need to set it when creating the composition even if it's the same url.

If webhook_event_url is not set for the composition, you can retrieve the composition with this request:

curl "https://api.telnyx.com/v2/room_compositions/<my-composition-id>" \
     -H 'Authorization: Bearer <my-telnyx-api-key>'

Once composition is completed, the response will have the AWS S3 download url for the resulting video.

Note: the AWS S3 download url is valid for an hour after it's generated.

Verified Calls Display Profile

Room Compositions

Telnyx Video Overview

HTTP ENDPOINTS

Video Room endpoints

Video Room Session endpoints

Video Room Participant endpoints

Getting Started with Video

Requirements

Configuration and Usage

Glossary

JavaScript Client SDK

Adding Telnyx to your JavaScript client application

Understanding the state of the video room

Publishing your local camera and mic stream

Knowing when a new participant joins the room

Subscribing to a remote stream

Knowing when a participant is speaking

Leaving the room

Integrating Chat message feature in your application

Receiving messages in Chat

Sending messages in Chat

Android Client SDK

Project structure

Adding the SDK to your Android client application

Before connecting to a Room

Get an API Key

Create a Room to join (if it doesn't exist)

Generate an a client token to join a room

Connect to Room

Publish video/audio stream

Remove video/audio track

Remove stream

Video/Audio Observables

State Observable

Event observables

Participants Observable

Joined Room Observable

Joined Participant Observable

Leaving participant id Observable

Connected to room Observable

Participant stream changed Observable

Stream status

Subscribe to a video stream

Remove subscription to a *video stream

Participant Talking Observable

Stats

Remote Video stats

Local Video stats

Remote audio stats

Local audio stats

iOS Client SDK

If you prefer to jump right in

An overview of the Video API

API of a Room

Events that are triggered in a Room

Understanding the state of the Room

Before getting started

Install the SDK

Get an API Key

Create a Room to join (if it doesn't exist)

Generate an a client token to join a room

Code Examples

Participating in a Room

Connect to a room

What is Room context?

Working with local media

Setting the quality of the local video

Publishing a stream

Unpublishing a stream

Working with Remote Participants and Streams

A remote participant who joins or leaves the room

Remote participants already in the room

Display a remote participant's media

Subscribing to a stream

Handling remote streams that are already publishing in the Room

Disconnecting from a Room

Starting a Video Composition

Create or update a room

Get the session id

Get recording ids

Create a video composition