> ## Documentation Index
> Fetch the complete documentation index at: https://developers.telnyx.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Observations

> The semantic observation model: observe the DOM, get element refs, interact by ref.

The API uses a **semantic observation model**: instead of clicking on pixel coordinates, you observe the DOM to get element references (`refs`), then use those refs to interact with elements.

## The Interaction Loop

```
navigate → observe → click / fill / select
    ▲           │             │
    │           │             │  uses observation_id
    │           │             │  + ref from observe
    └───────────┘             ▼
          (navigation invalidates observation)
```

1. **Navigate** to a URL within the session's `allowed_domains`.
2. **Observe** to capture a snapshot of the page — returns `observation_id` and `refs`.
3. **Interact** using the `observation_id` and a `ref` from step 2.

## Observe

```bash theme={null}
curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "observe"
  }'
```

```json theme={null}
{
  "data": {
    "status": "succeeded",
    "result": {
      "observation": {
        "observation_id": "obs_d9e5ab99-adf6-421e-b495-33f0ee727a52",
        "url": "https://example.com/",
        "title": "Example Domain",
        "refs": {
          "e1": { "role": "unknown", "name": "" },
          "e13": { "role": "link", "name": "Learn more" }
        },
        "tree": "- generic [active] [ref=e1]:\n  - link \"Learn more\" [ref=e13] [cursor=pointer]:\n    - /url: https://iana.org/help/example-domains",
        "truncated": false
      }
    }
  }
}
```

Full `tree` and `refs` are available in the original successful submission response when the action completes within `wait_seconds`. GET polling and idempotent replays return observation metadata with `tree: ""`, `refs: {}`, and `ref_count` instead of the full snapshot. After a 202, wait for the action to finish, then submit a fresh `observe` with `wait_seconds=25` to obtain usable refs. Do not reuse sample refs from this page.

### Response Fields

| Field | Description |
| - | - |
| `observation_id` | A unique identifier for this observation. Use it with `ref` in click/fill/select. |
| `refs` | A map of element references (e.g. `e1`, `e13`) with `role` and `name`. |
| `tree` | A human-readable text rendering of the DOM for inspection. |
| `truncated` | Whether the tree was truncated due to size limits. |

## Why observation\_id Expires

The `observation_id` is invalidated when the page navigates. After any `navigate` action (including a `click` that triggers a navigation), you must call `observe` again to get a fresh `observation_id` and `refs`.

Attempting to use a stale `observation_id` fails the action. The response is still HTTP 200 — the failure is on the action resource:

```json theme={null}
{
  "data": {
    "status": "failed",
    "error": { "code": "stale_observation", "retryable": false }
  }
}
```

## Interact by Ref

Once you have an `observation_id` and `refs`, interact with elements:

### Click

```bash theme={null}
curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "click",
    "observation_id": "obs_d9e5ab99-...",
    "ref": "e13"
  }'
```

### Fill (text input)

```bash theme={null}
curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "fill",
    "observation_id": "obs_...",
    "ref": "e5",
    "value": "user@example.com"
  }'
```

### Select (dropdown)

```bash theme={null}
curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "select",
    "observation_id": "obs_...",
    "ref": "e8",
    "value": "pro"
  }'
```

## include\_observation Shortcut

Use `include_observation: true` on a `click` or `navigate` action to get a fresh observation in the same response — saving a separate `observe` round-trip:

```bash theme={null}
curl -X POST https://api.telnyx.com/v2/browser_sessions/$SESSION_ID/actions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "click",
    "observation_id": "obs_...",
    "ref": "e2",
    "include_observation": true
  }'
```

The response will include a new `observation_id` and `refs` for the page that resulted from the click.