LiveKit Agents

The LiveKit Agent Framework is designed for building real-time, programmable participants that run on servers. The framework includes plugins for common workflows, such as voice activity detection and speech-to-text.

LiveKit has open-sourced some very helpful voice assistant examples. The following is an example from their repo, with slight modifications required to use Telnyx as your inference provider.

After cloning their repo, you can follow these instructions to run their minimal_assistant.py

Note

Make sure you have set the OPENAI_API_KEY environment variable to use your Telnyx API Key

python -m venv venv
source venv/bin/activate

pip install -r requirements.txt
pip install "livekit-plugins-elevenlabs>=0.7.1" 

python minimal_assistant.py download-files

Here's their minimal_assistant.py, modified to integrate with Telnyx for inference.

import asyncio

from dotenv import load_dotenv
from livekit import rtc
from livekit.agents import AutoSubscribe, JobContext, WorkerOptions, cli, llm
from livekit.agents.voice_assistant import VoiceAssistant
from livekit.plugins import deepgram, openai, silero, elevenlabs

load_dotenv()


async def entrypoint(ctx: JobContext):
    initial_ctx = llm.ChatContext().append(
        role="system",
        text=(
            "You are a voice assistant created by LiveKit. Your interface with users will be voice. "
            "You should use short and concise responses, and avoiding usage of unpronouncable punctuation."
        ),
    )

    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)

    # llm_plugin = openai.LLM()
    llm_plugin = openai.LLM(
        base_url="https://api.telnyx.com/v2/ai",
        model="meta-llama/Meta-Llama-3.1-70B-Instruct"
    )
    assistant = VoiceAssistant(
        vad=silero.VAD.load(),
        stt=deepgram.STT(),
        llm=llm_plugin,
        tts=elevenlabs.TTS(),
        chat_ctx=initial_ctx,
    )
    assistant.start(ctx.room)

    # listen to incoming chat messages, only required if you'd like the agent to
    # answer incoming messages from Chat
    chat = rtc.ChatManager(ctx.room)

    async def answer_from_text(txt: str):
        chat_ctx = assistant.chat_ctx.copy()
        chat_ctx.append(role="user", text=txt)
        stream = llm_plugin.chat(chat_ctx=chat_ctx)
        await assistant.say(stream)

    @chat.on("message_received")
    def on_chat_received(msg: rtc.ChatMessage):
        if msg.message:
            asyncio.create_task(answer_from_text(msg.message))

    await asyncio.sleep(1)
    await assistant.say("Hey, how can I help you today?", allow_interruptions=True)


if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

export LIVEKIT_URL=<your LiveKit server URL>
export LIVEKIT_API_KEY=<your API Key>
export LIVEKIT_API_SECRET=<your API Secret>
export ELEVEN_API_KEY=<your ElevenLabs API key>
export DEEPGRAM_API_KEY=<your Deepgram API key>
export OPENAI_API_KEY=<your **TELNYX** API key>

python minimal_assistant.py dev

LiveKit has built Agents Playground so you don't have to build your own frontend while you iterate on your agent.