AI Agent Usage

The iOS WebRTC SDK supports Voice AI Agent implementations.

To get started, follow the steps described here to build your first AI Assistant.

Once your AI Agent is up and running, you can use the SDK to communicate with your AI Agent with the following steps:

If you don't want to develop your own custom AI Agent interface from scratch, you can utilize our pre-developed AI Agent widget that provides a drop-in solution for voice AI interactions.

The iOS Telnyx Voice AI Widget is a standalone, embeddable widget that provides a complete voice AI assistant interface using the Telnyx WebRTC SDK.

Repository: https://github.com/team-telnyx/ios-telnyx-voice-ai-widget

Swift Package Manager: https://github.com/team-telnyx/ios-telnyx-voice-ai-widget.git

CocoaPods: pod 'TelnyxVoiceAIWidget', '~> 1.0.0'

• Drop-in Solution: Easy integration with minimal setup
• Multiple UI States: Collapsed, loading, expanded, and transcript views
• Icon-Only Mode: Floating action button-style interface for minimal UI footprint
• Audio Visualizer: Real-time audio visualization during conversations
• Theme Support: Light and dark theme compatibility
• Responsive Design: Optimized for various screen sizes
• Voice Controls: Mute/unmute and call management
• Transcript View: Full conversation history with text input
• Customizable Styling: Fine-tuned UI customization options

import TelnyxVoiceAIWidget

struct ContentView: View {
    @State private var showWidget = false

    var body: some View {
        AIAssistantWidget(
            assistantId: "your-assistant-id",
            shouldInitialize: showWidget
        )
    }
}

AIAssistantWidget(
    assistantId: "your-assistant-id",
    shouldInitialize: true,
    iconOnly: true // Enables floating action button mode
)

This widget handles all the complexity of AI Agent integration, providing a production-ready solution that you can customize to match your app's design.

This directory contains detailed documentation for AI Agent integration:

• Anonymous Login - How to connect to AI assistants without traditional authentication
• Starting Conversations - How to initiate calls with AI assistants
• Transcript Updates - Real-time conversation transcripts
• Text Messaging - Send text messages during active calls

1. Anonymous Login: Use anonymousLogin() to connect to your AI assistant
2. Start Conversation: Use newCall() to initiate a call (destination is ignored)
3. Receive Transcripts: Subscribe to transcript updates for real-time conversation updates
4. Send Text Messages: Use sendAIAssistantMessage() to send text during active calls

• No Authentication Required: Connect to AI assistants without SIP credentials
• Real-time Transcripts: Get live conversation updates with role identification
• Mixed Communication: Combine voice and text messaging in the same conversation
• Widget Settings: Access AI conversation configuration settings
• Standard Call Controls: Use existing call management methods (mute, hold, end call)

• After anonymousLogin(), all subsequent calls are routed to the specified AI assistant
• Transcript functionality is only available for AI assistant conversations
• AI assistants automatically answer calls - no manual answer required
• Text messages appear in transcript updates alongside spoken conversation