Observations

The API uses a semantic observation model: instead of clicking on pixel coordinates, you observe the DOM to get element references (refs), then use those refs to interact with elements.

The Interaction Loop

navigate → observe → click / fill / select
    ▲           │             │
    │           │             │  uses observation_id
    │           │             │  + ref from observe
    └───────────┘             ▼
          (navigation invalidates observation)

Navigate to a URL within the session’s allowed_domains.
Observe to capture a snapshot of the page — returns observation_id and refs.
Interact using the observation_id and a ref from step 2.

Observe

curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "observe"
  }'

{
  "data": {
    "status": "succeeded",
    "result": {
      "observation": {
        "observation_id": "obs_d9e5ab99-adf6-421e-b495-33f0ee727a52",
        "url": "https://example.com/",
        "title": "Example Domain",
        "refs": {
          "e1": { "role": "unknown", "name": "" },
          "e13": { "role": "link", "name": "Learn more" }
        },
        "tree": "- generic [active] [ref=e1]:\n  - link \"Learn more\" [ref=e13] [cursor=pointer]:\n    - /url: https://iana.org/help/example-domains",
        "truncated": false
      }
    }
  }
}

Full tree and refs are available in the original successful submission response when the action completes within wait_seconds. GET polling and idempotent replays return observation metadata with tree: "", refs: {}, and ref_count instead of the full snapshot. After a 202, wait for the action to finish, then submit a fresh observe with wait_seconds=25 to obtain usable refs. Do not reuse sample refs from this page.

Response Fields

Field	Description
`observation_id`	A unique identifier for this observation. Use it with `ref` in click/fill/select.
`refs`	A map of element references (e.g. `e1`, `e13`) with `role` and `name`.
`tree`	A human-readable text rendering of the DOM for inspection.
`truncated`	Whether the tree was truncated due to size limits.

Why observation_id Expires

The observation_id is invalidated when the page navigates. After any navigate action (including a click that triggers a navigation), you must call observe again to get a fresh observation_id and refs. Attempting to use a stale observation_id fails the action. The response is still HTTP 200 — the failure is on the action resource:

{
  "data": {
    "status": "failed",
    "error": { "code": "stale_observation", "retryable": false }
  }
}

Interact by Ref

Once you have an observation_id and refs, interact with elements:

Click

curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "click",
    "observation_id": "obs_d9e5ab99-...",
    "ref": "e13"
  }'

Fill (text input)

curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "fill",
    "observation_id": "obs_...",
    "ref": "e5",
    "value": "[email protected]"
  }'

curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "select",
    "observation_id": "obs_...",
    "ref": "e8",
    "value": "pro"
  }'

include_observation Shortcut

Use include_observation: true on a click or navigate action to get a fresh observation in the same response — saving a separate observe round-trip:

curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "click",
    "observation_id": "obs_...",
    "ref": "e2",
    "include_observation": true
  }'

The response will include a new observation_id and refs for the page that resulted from the click.

Agent Browser

Features

API Reference

For AI Agents

The Interaction Loop

Observe

Response Fields

Why observation_id Expires

Interact by Ref

Click

Fill (text input)

include_observation Shortcut

​The Interaction Loop

​Observe

​Response Fields

​Why observation_id Expires

​Interact by Ref

​Click

​Fill (text input)

​Select (dropdown)

​include_observation Shortcut

The Interaction Loop

Observe

Response Fields

Why observation_id Expires

Interact by Ref

Click

Fill (text input)

Select (dropdown)

include_observation Shortcut