Skip to main content

​
Sending Text Messages to AI Agents

​
Overview

In addition to voice conversation, you can send text messages directly to the AI Agent during an active call. This allows for mixed-mode communication where users can both speak and type messages to the AI Assistant.

​
Method Signature

void sendConversationMessage(String message, {String? base64Image})
This method is available on the Call object during an active AI Assistant conversation.

​
Parameters

  • message (String): The text message to send to the AI Agent
  • base64Image (String?, optional): A base64 encoded image string for vision-capable AI models
Note: To provide images to your AI assistant, ensure you are using a vision-capable model. The base64Image parameter accepts either:
  • A complete data URL format: data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQ...
  • A raw base64 string (will be automatically formatted as JPEG): /9j/4AAQSkZJRgABAQAAAQ...

​
Basic Usage

​
Getting the Active Call

// Get the active call instance (after successfully connecting and calling)
Call? activeCall = _telnyxClient.calls.values.firstOrNull;

if (activeCall != null) {
  // Send a text message to the AI Agent
  activeCall.sendConversationMessage("Hello, can you help me with my account?");
  
  // Send a message with an image (for vision-capable models)
  activeCall.sendConversationMessage(
    "Can you analyze this image?", 
    base64Image: "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQ..."
  );
  
  // Send only an image without text
  activeCall.sendConversationMessage(
    "", 
    base64Image: "/9j/4AAQSkZJRgABAQAAAQ..." // Raw base64 (auto-formatted as JPEG)
  );
}

​
Important Notes

  • AI Assistant Only: The sendConversationMessage method is only available during AI Assistant conversations
  • Transcript Integration: Text messages sent this way will appear in the transcript updates alongside spoken conversation
  • Processing: The AI Agent will process and respond to text messages just like spoken input
  • Active Call Required: You must have an active call established before sending text messages
  • Real-time: Messages are sent in real-time and will be processed immediately by the AI agent
  • Vision Models: Base64 image support requires vision-capable AI models to be configured in your AI Assistant
  • Image Formats: Supported image formats include JPEG, PNG, GIF, and WebP
  • Image Size: Consider image file size and compression for optimal performance
  • Multimodal: You can send text and images together, or send images without text for pure visual analysis

​
Troubleshooting

​
Message Not Sent

If messages aren’t being sent:
  1. Verify the call is active (call.callState.isActive)
  2. Check that you’re in an AI Assistant conversation (after anonymousLogin)
  3. Ensure the call object is not null
  4. Verify network connectivity

​
Messages Not Appearing in Transcript

If sent messages don’t appear in the transcript:
  1. Confirm onTranscriptUpdate callback is properly set
  2. Check that the message was successfully sent
  3. Wait a moment for the transcript to update

​
Next Steps