Skip to main content

Getting started with Telnyx Inference API


Welcome to the Telnyx Inference API! This guide will walk you through the basics of chatting with open-source language models running on Telnyx GPUs.


Python Example

Let's complete your first chat. Here's some simple Python to interact with a language model:

import os
from openai import OpenAI

client = OpenAI(

chat_completion =
"role": "user",
"content": "Can you explain 10DLC to me?"

for chunk in chat_completion:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)

Note: Make sure you have set the TELNYX_API_KEY environment variable

Core Concepts

  • Messages: This refers to the history of messages in a chat
  • Roles: Every message has a role: system, user, assistant, or tool.
    • System messages are usually sent once at the start of a chat, and influence the entire chat
    • User messages refer to what the end user has input
    • Assistant messages refer to what the model has output
    • Tool messages refer to results of any tool calls
  • Models: In the context of chat completions, we are talking about large language models (LLMs). Your choice of LLM will affect the quality, speed, and price of your chat completions.
  • Streaming: For real-time interactions, you will want the ability to stream partial responses back to a client as they are completed. To achieve this, we follow the same Server-sent events standard as OpenAI.

Next Steps


Have questions or need help troubleshooting? Our support team is here to assist you. Join our Slack community to connect with other developers and the Telnyx team.

On this page