Calling LLMs

The Agent SDK does not run a model and does not impose an agent framework. The loop that does the thinking — the harness — is yours to choose; the SDK supplies the durable substrate — history, timers, state — and this.messages converts that history into the format your LLM stack expects. There are two broad ways to wire a harness in.

Roll your own loop

Your process() method is the agent loop: build the message list, make one call, handle the reply. The Telnyx API binding is a pre-authenticated client — declare [telnyx] in telnyx.toml and inference is a method call, no API key to manage. this.messages.toOpenAI() produces exactly the payload it takes:

async process(): Promise<void> {
  const history = await this.messages.toOpenAI();

  // An API error rejects → the task fails → the scheduler retries
  const res = await this.env.TELNYX.ai.openai.chat.createCompletion({
    model: "zai-org/GLM-5.2",
    messages: [{ role: "system", content: SYSTEM_PROMPT }, ...history],
  });
  await this.messages.add("assistant", res.choices[0].message.content);
}

Bringing a different provider? Any OpenAI-compatible endpoint works over fetch — swap the URL, model, and a key held in a secret. The official openai and @anthropic-ai/sdk clients work too, fed by toOpenAI() and toAnthropic(). → Full example: Roll Your Own Agent

Bring a framework

Any agent framework that runs on Node — LangGraph, LangChain, and friends — runs inside process(), with the SDK as its durable memory. toLangChain() returns plain { role, content } messages, which LangGraph accepts as-is:

const llm = new ChatOpenAI({
  model: "zai-org/GLM-5.2",
  apiKey: process.env.TELNYX_API_KEY,
  configuration: { baseURL: "https://api.telnyx.com/v2/ai/openai" },
});
const supportAgent = createReactAgent({ llm, tools: [lookupOrder] });

async process(): Promise<void> {
  const history = await this.messages.toLangChain();
  const out = await supportAgent.invoke({ messages: history });
  await this.messages.add("assistant", String(out.messages.at(-1)?.content ?? ""));
}

The framework owns the reasoning loop and tool calls; the actor owns durability, retries, and follow-up timers. → Full example: LangGraph Agent

Where to make the call

Make LLM calls from a queued or scheduled task, not from the inbound method. Inbound RPC runs under a 30-second budget; tasks run in the actor’s alarm handler with a budget on the order of minutes, and a thrown error triggers retry with backoff instead of a lost webhook. See How Agents Run.

Get Started

Concepts

Build

Examples

Reference

For AI Agents

Roll your own loop

Bring a framework

Where to make the call

​Roll your own loop

​Bring a framework

​Where to make the call

Roll your own loop

Bring a framework

Where to make the call