Talk to an AI Assistant over WebSocket
Real-time voice conversation with a Telnyx AI Assistant. The client streams PCM16 audio to Telnyx and receives lifecycle, transcription, and assistant-audio frames in return.
Authentication
Requires authentication via a Bearer token (Telnyx API v2 key).
{
"type": "input_audio_buffer.append",
"audio": "eW91ciBiYXNlNjQgZW5jb2RlZCBwY20xNiBhdWRpbw=="
}{
"type": "session.created",
"session": {
"conversation_id": "3E6F995F-85F7-4705-9741-53B116D28237",
"assistant_id": "assistant-0f4e8b2a",
"audio": {
"input": {
"format": {
"type": "audio/pcm",
"rate": 16000
},
"turn_detection": {
"type": "server_vad"
}
},
"output": {
"format": {
"type": "audio/pcm",
"rate": 24000
}
}
}
}
}{
"type": "input_audio_buffer.speech_started"
}{
"type": "input_audio_buffer.speech_stopped"
}{
"type": "conversation.item.input_audio_transcription.completed",
"transcript": "What are your business hours?"
}{
"type": "response.created",
"response": {
"id": "resp_9c1f4a2b"
}
}{
"type": "response.output_audio.delta",
"response_id": "resp_9c1f4a2b",
"item_id": "item_5d2e7f10",
"delta": "eW91ciBiYXNlNjQgZW5jb2RlZCBwY20xNiBhdWRpbw=="
}{
"type": "response.output_audio_transcript.delta",
"response_id": "resp_9c1f4a2b",
"item_id": "item_5d2e7f10",
"delta": "We are open from 9am "
}{
"type": "response.output_audio.done",
"response_id": "resp_9c1f4a2b"
}{
"type": "response.done",
"response": {
"id": "resp_9c1f4a2b",
"status": "completed"
}
}{
"type": "error",
"error": {
"code": "invalid_audio",
"message": "Invalid base64-encoded audio payload"
}
}Identifier of the AI Assistant to converse with.
Telnyx API v2 Bearer token authentication.
Query parameters passed when opening the WebSocket connection.
Client-to-Telnyx frame carrying a chunk of microphone audio.
The audio field is a base64-encoded PCM16 payload at the sample rate requested with the input_sample_rate query parameter. Frames larger than 1 MiB are rejected with a frame_too_large error.
First frame sent by Telnyx once the conversation has started. Confirms the session and the negotiated input and output audio formats.
Sent when server-side voice-activity detection determines the user has started speaking. Edge-triggered.
Sent when server-side voice-activity detection determines the user has stopped speaking. Edge-triggered, and never sent without a preceding speech-started.
Transcript of the audio the user streamed for the current turn.
Marks the start of an assistant turn. The response.id correlates the audio and transcript deltas that follow.
A chunk of the assistant''s synthesized speech, as base64-encoded PCM16 at the output rate announced in session.created.
A chunk of the text transcript of the assistant''s spoken response, streamed alongside the audio deltas.
Sent when the assistant has finished streaming audio for the turn.
Marks the end of an assistant turn, including its final status.
Sent when an error occurs. Some errors are non-fatal and leave the session open; others close the connection after the frame is sent.
Was this page helpful?
{
"type": "input_audio_buffer.append",
"audio": "eW91ciBiYXNlNjQgZW5jb2RlZCBwY20xNiBhdWRpbw=="
}{
"type": "session.created",
"session": {
"conversation_id": "3E6F995F-85F7-4705-9741-53B116D28237",
"assistant_id": "assistant-0f4e8b2a",
"audio": {
"input": {
"format": {
"type": "audio/pcm",
"rate": 16000
},
"turn_detection": {
"type": "server_vad"
}
},
"output": {
"format": {
"type": "audio/pcm",
"rate": 24000
}
}
}
}
}{
"type": "input_audio_buffer.speech_started"
}{
"type": "input_audio_buffer.speech_stopped"
}{
"type": "conversation.item.input_audio_transcription.completed",
"transcript": "What are your business hours?"
}{
"type": "response.created",
"response": {
"id": "resp_9c1f4a2b"
}
}{
"type": "response.output_audio.delta",
"response_id": "resp_9c1f4a2b",
"item_id": "item_5d2e7f10",
"delta": "eW91ciBiYXNlNjQgZW5jb2RlZCBwY20xNiBhdWRpbw=="
}{
"type": "response.output_audio_transcript.delta",
"response_id": "resp_9c1f4a2b",
"item_id": "item_5d2e7f10",
"delta": "We are open from 9am "
}{
"type": "response.output_audio.done",
"response_id": "resp_9c1f4a2b"
}{
"type": "response.done",
"response": {
"id": "resp_9c1f4a2b",
"status": "completed"
}
}{
"type": "error",
"error": {
"code": "invalid_audio",
"message": "Invalid base64-encoded audio payload"
}
}