Migrate from the OpenAI Realtime API

With OpenAI, the socket is the product: you configure the session, manage the conversation, and request every response. With Telnyx, the AI Assistant is the product: instructions, model, voice, and tools live on the assistant, and the assistant owns turn-taking. The socket only carries the conversation.

​

Before you start

​

Connection and authentication

import WebSocket from "ws";

const ws = new WebSocket("wss://api.openai.com/v1/realtime?model=gpt-realtime", {
  headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
});

ws.on("open", () => {
  ws.send(JSON.stringify({
    type: "session.update",
    session: {
      type: "realtime",
      instructions: "You are a helpful assistant for Acme Inc.",
      audio: {
        input: { format: { type: "audio/pcm", rate: 24000 } },
        output: { voice: "marin" },
      },
    },
  }));
});

import WebSocket from "ws";

const ASSISTANT_ID = "assistant-0f4e8b2a"; // instructions, model, voice, tools already configured

const ws = new WebSocket(
  `wss://api.telnyx.com/v2/ai/assistants/${ASSISTANT_ID}/conversation?input_sample_rate=24000`,
  { headers: { Authorization: `Bearer ${process.env.TELNYX_API_KEY}` } }
);

// No session.update — wait for session.created and start streaming audio.

case "session.created":
  outputRate = event.session.audio.output.format.rate;
  break;

​

Event mapping

​

Client → server frames

​

Server → client frames

​

Turn-taking: delete your response orchestration

sendAudioChunks(ws, chunks);
ws.send(JSON.stringify({ type: "input_audio_buffer.commit" }));
ws.send(JSON.stringify({ type: "response.create" }));

sendAudioChunks(ws, chunks);
// That's it. Server VAD detects the end of speech and the assistant responds.

• Turn detection is always server_vad. You cannot disable it (turn_detection: null) or use semantic VAD. Push-to-talk UIs still work — only send audio while the button is held — but the turn boundary is still decided by server VAD, tuned via Interruption Settings.
• Automatic responses cannot be turned off. There is no create_response: false mode where you inspect the transcript before allowing a reply.
• Text turns need no trigger. conversation.item.create with input_text gets an automatic spoken response — do not follow it with response.create.

​

Interruption handling

case "input_audio_buffer.speech_started": {
  playback.length = 0;
  ws.send(JSON.stringify({
    type: "conversation.item.truncate",
    item_id: currentItemId,
    content_index: 0,
    audio_end_ms: msPlayedSoFar,
  }));
  break;
}

case "input_audio_buffer.speech_started":
  playback.length = 0; // just stop playing — Telnyx handles the rest
  break;

​

Function calling

• Tools your backend served (the common OpenAI pattern) usually become webhook or MCP tools: Telnyx calls your endpoint directly and the socket only shows informational response.tool_call.started / .completed frames. Your client-side function-calling code is deleted entirely.
• Tools that must run in your client process become client-side tools. The round-trip resembles OpenAI’s, with two changes: the request arrives as a single conversation.item.created frame with complete arguments (no function_call_arguments.delta streaming, no digging through response.done output), and you don’t send response.create after the output.

case "response.done": {
  for (const item of event.response.output ?? []) {
    if (item.type === "function_call") {
      const result = await runTool(item.name, JSON.parse(item.arguments));
      ws.send(JSON.stringify({
        type: "conversation.item.create",
        item: { type: "function_call_output", call_id: item.call_id, output: JSON.stringify(result) },
      }));
      ws.send(JSON.stringify({ type: "response.create" }));
    }
  }
  break;
}

case "conversation.item.created": {
  if (event.item.type === "function_call") {
    const result = await runTool(event.item.name, JSON.parse(event.item.arguments));
    ws.send(JSON.stringify({
      type: "conversation.item.create",
      item: { type: "function_call_output", call_id: event.item.call_id, output: JSON.stringify(result) },
    }));
    // No response.create — the assistant continues automatically.
  }
  break;
}

​

OpenAI features without a Telnyx equivalent

• Session reconfiguration — no session.update; configuration is fixed for the life of the connection.
• Manual turn control — no turn_detection: null, create_response: false, input_audio_buffer.commit/.clear, or semantic VAD.
• Out-of-band responses and custom context — no response.create, so no conversation: "none", per-response input arrays, response metadata, or per-response overrides (voice, modality, max_output_tokens).
• Text-only output — responses are always spoken; use the transcript deltas for text. (For a pure text channel, use the Assistants chat API instead — see Assistants API in the sidebar.)
• Conversation item manipulation — no conversation.item.truncate, .retrieve, or .delete, and no assistant-message injection into history.
• Image input — input_image content is not supported; conversation.item.create accepts input_text and function_call_output items only (anything else is rejected with invalid_item).
• G.711 audio — PCM16 only, in both directions.
• Rate-limit telemetry — no rate_limits.updated frames.

​

Migration checklist

1. Create an assistant and move session.update contents into its configuration (instructions, model, voice, tools, transcription, interruption).
2. Swap the URL and API key; pick input_sample_rate via query parameter.
3. Read the output sample rate from session.created instead of assuming 24 kHz.
4. Delete: session.update, response.create, input_audio_buffer.commit/.clear, conversation.item.truncate, and truncation bookkeeping.
5. Rewire function calling: backend tools → webhook/MCP tools (delete client code); in-client tools → handle conversation.item.created function_call items.
6. Keep: audio append loop, playback-flush on speech_started, response.cancel for programmatic interrupts, transcript rendering from .delta frames.
7. Update error handling to the Telnyx error codes, and treat a reconnect as a new conversation.
8. Test barge-in, tool calls, and long-silence behavior (session_idle_timeout) end to end.

​

Learn more

• Realtime voice conversations over WebSocket — The full guide to this API
• Conversation WebSocket reference — The complete frame-by-frame reference, under Assistants API → Conversation WebSocket in the sidebar
• Client-Side Tools — Tool handlers that run in your application
• Custom LLM — Bring your own model to a Telnyx assistant