> ## Documentation Index
> Fetch the complete documentation index at: https://developers.telnyx.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Migrate from the OpenAI Realtime API
> Move a realtime voice application from the OpenAI Realtime API to the Telnyx Assistant Conversation WebSocket: endpoint and auth changes, an event-by-event mapping, and what to delete from your client.
The Telnyx [Assistant Conversation WebSocket](/docs/inference/ai-assistants/realtime-conversations) speaks a wire format intentionally close to the OpenAI Realtime API over WebSockets: JSON frames with a `type` field, base64 PCM16 audio in `input_audio_buffer.append`, assistant speech in `response.output_audio.delta`, and familiar event names throughout. Most migrations are a matter of changing the URL, moving session configuration onto the assistant, and deleting client code that Telnyx makes unnecessary.
The one architectural shift to internalize before touching code:
> **With OpenAI, the socket is the product: you configure the session, manage the conversation, and request every response. With Telnyx, the [AI Assistant](https://portal.telnyx.com/#/ai/assistants) is the product: instructions, model, voice, and tools live on the assistant, and the assistant owns turn-taking. The socket only carries the conversation.**
This makes the Telnyx client simpler than the OpenAI client it replaces — most of the migration is deleting code.
## Before you start
Create and configure an AI Assistant in the [Portal](https://portal.telnyx.com/#/ai/assistants) (or via the Assistants API — see **Assistants API** in the sidebar). Everything you used to send in `session.update` becomes assistant configuration:
| In your OpenAI `session.update` | On the Telnyx assistant |
| -------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `instructions` | The assistant's **Instructions** |
| Model selection | The assistant's **Model** (multiple LLM providers, or [bring your own](/docs/inference/ai-assistants/custom-llm)) |
| `audio.output.voice` | The assistant's **Voice** |
| `tools`, `tool_choice` | The assistant's **Tools** ([webhook, MCP](/docs/inference/ai-assistants/tools-library), or [client-side](/docs/inference/ai-assistants/client-side-tools)) |
| `audio.input.transcription` | [Transcription Settings](/docs/inference/ai-assistants/transcription-settings) |
| `audio.input.turn_detection` tuning | [Interruption Settings](/docs/inference/ai-assistants/interruption-settings) — VAD itself is always server-side |
| `audio.input.format` / `audio.output.format` | Query parameters at connect time (PCM16 only) |
Nothing can be reconfigured over the socket mid-conversation — there is no `session.update` frame. If your application changes instructions or tools mid-session today, restructure that into assistant configuration ([dynamic variables](/docs/inference/ai-assistants/dynamic-variables), [memory](/docs/inference/ai-assistants/memory)) before migrating.
## Connection and authentication
| | OpenAI Realtime | Telnyx |
| --------------------- | -------------------------------------------- | ------------------------------------------------------------------- |
| URL | `wss://api.openai.com/v1/realtime?model=...` | `wss://api.telnyx.com/v2/ai/assistants/{assistant_id}/conversation` |
| Auth | `Authorization: Bearer` OpenAI key | `Authorization: Bearer` Telnyx API v2 key |
| What the path selects | A model | An assistant (model, voice, instructions, tools included) |
| Audio negotiation | `session.update` after connect | Query parameters at connect time |
```javascript Before (OpenAI) theme={null}
import WebSocket from "ws";
const ws = new WebSocket("wss://api.openai.com/v1/realtime?model=gpt-realtime", {
headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
});
ws.on("open", () => {
ws.send(JSON.stringify({
type: "session.update",
session: {
type: "realtime",
instructions: "You are a helpful assistant for Acme Inc.",
audio: {
input: { format: { type: "audio/pcm", rate: 24000 } },
output: { voice: "marin" },
},
},
}));
});
```
```javascript After (Telnyx) theme={null}
import WebSocket from "ws";
const ASSISTANT_ID = "assistant-0f4e8b2a"; // instructions, model, voice, tools already configured
const ws = new WebSocket(
`wss://api.telnyx.com/v2/ai/assistants/${ASSISTANT_ID}/conversation?input_sample_rate=24000`,
{ headers: { Authorization: `Bearer ${process.env.TELNYX_API_KEY}` } }
);
// No session.update — wait for session.created and start streaming audio.
```
Audio is PCM16-only in both directions (`g711_ulaw`/`g711_alaw` are not available over this WebSocket). Input sample rate is one of `8000`, `16000`, `24000`, `44100`, `48000` Hz (default `16000`). The output rate is determined by the assistant's voice — read it from `session.created` instead of assuming 24 kHz:
```javascript theme={null}
case "session.created":
outputRate = event.session.audio.output.format.rate;
break;
```
## Event mapping
### Client → server frames
| OpenAI Realtime | Telnyx | Notes |
| --------------------------------------------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `input_audio_buffer.append` | `input_audio_buffer.append` | Identical shape: `{ "audio": "" }`. Frames over 1 MiB are rejected; stream at a real-time pace. |
| `input_audio_buffer.commit` | — | Delete. Server VAD ends turns automatically. |
| `input_audio_buffer.clear` | — | Delete. No manual buffer management. |
| `conversation.item.create` (user text) | `conversation.item.create` | Same shape with `input_text` content. The assistant answers automatically. |
| `conversation.item.create` (`function_call_output`) | `conversation.item.create` | Same shape: `{ "call_id": ..., "output": ... }`. |
| `conversation.item.truncate` | — | No equivalent. Flush your local playback queue on `input_audio_buffer.speech_started` instead (see below). |
| `conversation.item.retrieve` / `conversation.item.delete` | — | No equivalent. |
| `response.create` | — | Delete. The assistant owns turn-taking; there is no frame to request a response. |
| `response.cancel` | `response.cancel` | Same, including optional `response_id`. |
| `session.update` | — | Delete. Configure the assistant instead (see above). |
| `output_audio_buffer.clear` (WebRTC) | — | Not applicable. |
### Server → client frames
| OpenAI Realtime | Telnyx | Notes |
| ----------------------------------------------------------------------------------- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `session.created` | `session.created` | Different payload: carries `conversation_id`, `assistant_id`, and the negotiated audio formats. |
| `session.updated` | — | Never sent (nothing to update). |
| `input_audio_buffer.speech_started` / `speech_stopped` | Same names | Edge-triggered; payload has no `audio_start_ms` or `item_id`. |
| `input_audio_buffer.committed` | — | Never sent. |
| `conversation.item.input_audio_transcription.completed` | Same name | Payload is just `{ "transcript": "..." }`. No `.delta` / `.failed` variants. |
| `conversation.item.added` / `.done` | — | Never sent. |
| `conversation.item.created` | `conversation.item.created` | Sent only to request a **client-side tool call** (a `function_call` item). |
| `response.created` | `response.created` | Payload is `{ "response": { "id": ... } }`. |
| `response.output_item.added` / `response.content_part.added` (and `.done` variants) | — | Never sent. A turn is delimited by `response.created` … `response.done`. |
| `response.output_audio.delta` / `.done` | Same names | Deltas carry `response_id` and `item_id` for correlation. |
| `response.output_audio_transcript.delta` | Same name | No `.done` variant — treat `response.output_audio.done` as end of the turn's streaming. |
| `response.output_text.delta` / `.done` | — | Never sent. Responses are always spoken; the transcript stream is the text. |
| `response.function_call_arguments.delta` / `.done` | — | Replaced by a single `conversation.item.created` frame with complete `arguments`. |
| `response.done` | `response.done` | Payload is `{ "response": { "id": ..., "status": "completed" \| "cancelled" } }`. |
| `rate_limits.updated` | — | Never sent. |
| `error` | `error` | Same shape (`error.code`, `error.message`); Telnyx-specific codes — see [Error handling](/docs/inference/ai-assistants/realtime-conversations#error-handling). |
| — | `response.tool_call.started` / `.completed` | New, informational: server-side (webhook/MCP) tool activity. Display it if useful; never respond to it. |
## Turn-taking: delete your response orchestration
On OpenAI, a manual-VAD or push-to-talk client drives the conversation: `append` → `commit` → `response.create`. On Telnyx the server drives it — you stream audio continuously and turns happen:
```javascript Before (OpenAI, manual turns) theme={null}
sendAudioChunks(ws, chunks);
ws.send(JSON.stringify({ type: "input_audio_buffer.commit" }));
ws.send(JSON.stringify({ type: "response.create" }));
```
```javascript After (Telnyx) theme={null}
sendAudioChunks(ws, chunks);
// That's it. Server VAD detects the end of speech and the assistant responds.
```
Consequences to plan for:
* **Turn detection is always `server_vad`.** You cannot disable it (`turn_detection: null`) or use semantic VAD. Push-to-talk UIs still work — only send audio while the button is held — but the turn boundary is still decided by server VAD, tuned via [Interruption Settings](/docs/inference/ai-assistants/interruption-settings).
* **Automatic responses cannot be turned off.** There is no `create_response: false` mode where you inspect the transcript before allowing a reply.
* **Text turns need no trigger.** `conversation.item.create` with `input_text` gets an automatic spoken response — do not follow it with `response.create`.
## Interruption handling
Barge-in is automatic: when the user speaks over the assistant, Telnyx cancels the response (final `response.done` has `status: "cancelled"`) and starts a new turn. Your OpenAI truncation bookkeeping — tracking `item_id`, measuring played milliseconds, sending `conversation.item.truncate` — has no Telnyx equivalent and should be deleted. Keep exactly one client-side behavior: flush locally queued audio when speech starts.
```javascript Before (OpenAI) theme={null}
case "input_audio_buffer.speech_started": {
playback.length = 0;
ws.send(JSON.stringify({
type: "conversation.item.truncate",
item_id: currentItemId,
content_index: 0,
audio_end_ms: msPlayedSoFar,
}));
break;
}
```
```javascript After (Telnyx) theme={null}
case "input_audio_buffer.speech_started":
playback.length = 0; // just stop playing — Telnyx handles the rest
break;
```
## Function calling
Tool definitions move from `session.update` payloads to the assistant's tool configuration. At runtime, two OpenAI patterns collapse into one Telnyx pattern:
* **Tools your backend served** (the common OpenAI pattern) usually become **webhook or MCP tools**: Telnyx calls your endpoint directly and the socket only shows informational `response.tool_call.started` / `.completed` frames. Your client-side function-calling code is deleted entirely.
* **Tools that must run in your client process** become [client-side tools](/docs/inference/ai-assistants/client-side-tools). The round-trip resembles OpenAI's, with two changes: the request arrives as a single `conversation.item.created` frame with complete arguments (no `function_call_arguments.delta` streaming, no digging through `response.done` output), and you don't send `response.create` after the output.
```javascript Before (OpenAI) theme={null}
case "response.done": {
for (const item of event.response.output ?? []) {
if (item.type === "function_call") {
const result = await runTool(item.name, JSON.parse(item.arguments));
ws.send(JSON.stringify({
type: "conversation.item.create",
item: { type: "function_call_output", call_id: item.call_id, output: JSON.stringify(result) },
}));
ws.send(JSON.stringify({ type: "response.create" }));
}
}
break;
}
```
```javascript After (Telnyx) theme={null}
case "conversation.item.created": {
if (event.item.type === "function_call") {
const result = await runTool(event.item.name, JSON.parse(event.item.arguments));
ws.send(JSON.stringify({
type: "conversation.item.create",
item: { type: "function_call_output", call_id: event.item.call_id, output: JSON.stringify(result) },
}));
// No response.create — the assistant continues automatically.
}
break;
}
```
## OpenAI features without a Telnyx equivalent
Audit your application for these before migrating — they are not available over the Assistant Conversation WebSocket today:
* **Session reconfiguration** — no `session.update`; configuration is fixed for the life of the connection.
* **Manual turn control** — no `turn_detection: null`, `create_response: false`, `input_audio_buffer.commit`/`.clear`, or semantic VAD.
* **Out-of-band responses and custom context** — no `response.create`, so no `conversation: "none"`, per-response `input` arrays, response `metadata`, or per-response overrides (voice, modality, `max_output_tokens`).
* **Text-only output** — responses are always spoken; use the transcript deltas for text. (For a pure text channel, use the Assistants chat API instead — see **Assistants API** in the sidebar.)
* **Conversation item manipulation** — no `conversation.item.truncate`, `.retrieve`, or `.delete`, and no assistant-message injection into history.
* **Image input** — `input_image` content is not supported; `conversation.item.create` accepts `input_text` and `function_call_output` items only (anything else is rejected with `invalid_item`).
* **G.711 audio** — PCM16 only, in both directions.
* **Rate-limit telemetry** — no `rate_limits.updated` frames.
If one of these is load-bearing for your application, talk to your Telnyx point of contact before scheduling the migration.
## Migration checklist
1. Create an assistant and move `session.update` contents into its configuration (instructions, model, voice, tools, transcription, interruption).
2. Swap the URL and API key; pick `input_sample_rate` via query parameter.
3. Read the output sample rate from `session.created` instead of assuming 24 kHz.
4. Delete: `session.update`, `response.create`, `input_audio_buffer.commit`/`.clear`, `conversation.item.truncate`, and truncation bookkeeping.
5. Rewire function calling: backend tools → webhook/MCP tools (delete client code); in-client tools → handle `conversation.item.created` `function_call` items.
6. Keep: audio append loop, playback-flush on `speech_started`, `response.cancel` for programmatic interrupts, transcript rendering from `.delta` frames.
7. Update error handling to the [Telnyx error codes](/docs/inference/ai-assistants/realtime-conversations#error-handling), and treat a reconnect as a new conversation.
8. Test barge-in, tool calls, and long-silence behavior (`session_idle_timeout`) end to end.
## Learn more
* **[Realtime voice conversations over WebSocket](/docs/inference/ai-assistants/realtime-conversations)** — The full guide to this API
* **Conversation WebSocket reference** — The complete frame-by-frame reference, under **Assistants API → Conversation WebSocket** in the sidebar
* **[Client-Side Tools](/docs/inference/ai-assistants/client-side-tools)** — Tool handlers that run in your application
* **[Custom LLM](/docs/inference/ai-assistants/custom-llm)** — Bring your own model to a Telnyx assistant