Skip to main content

Prerequisites

Install the OpenAI SDK:
The Inference API is OpenAI-compatible. Any OpenAI SDK works with a base_url swap.

Python

Reasoning models such as zai-org/GLM-5.2 return their chain-of-thought in a separate reasoning_content field (on message for non-streaming responses, or delta when streaming). Models without reasoning simply omit it, so the getattr(..., "reasoning_content", None) guard works for every model.

Core Concepts

Messages

Chat history passed to the model.

Roles

Every message has a role: system, user, assistant, or tool.
  • system — model behavior instructions
  • user — end-user input
  • assistant — model output
  • tool — function call results. See Function Calling.

Models

Available Models lists all hosted LLMs with context lengths and capabilities.

Streaming

Server-sent events, same as OpenAI.

What Next?