Test and Debug

Two commands answer the questions a change raises: traces shows what one run did, step by step, and eval scores saved test conversations so a prompt or model change is checked against the same cases every time. Both work against dev and against the deployed agent.

​

The Loop

1. Run the agent on this machine:
   telnyx-acp dev
   
2. Ask it something, and read what it did:
   telnyx-acp invoke support-agent --conversation alice --local --trace "What's the weather in Paris?"
   
   It's 14°C and cloudy in Paris right now.
   run run-7f3e…  invoke  completed  3.6 s  4,803 tok  1 tool call
   ├─     +6 ms  model  meta-llama/Llama-3.3-70B-Instruct  step 1  1.3 s  2,333 tok  tool-calls
   ├─    +1.3 s  tool  web_search                          ok  772 ms  408 B
   ├─    +2.1 s  model  meta-llama/Llama-3.3-70B-Instruct  step 2  1.5 s  2,470 tok  stop
   └─    +3.6 s  deliver  invoke                           ok  0 ms
   
   A slow reply shows where the time went; a wrong one shows which tool ran, or that none did.
3. Save what it should do as cases, in evals/:
   {"name": "weather", "input": "What's the weather in Paris?", "expect": {"tool": "web_search", "maxMs": 10000}}
   {"name": "refunds", "input": "Can I get a refund?", "expect": {"rubric": [{"name": "policy", "criteria": "States the 30-day refund window"}, {"name": "no-promise", "criteria": "Does not promise a refund"}]}}
   
4. Run them against dev while changing instructions.md:
   telnyx-acp eval --local
   
5. Deploy, then run the same cases against the deployed agent, in CI if the project has it:
   telnyx-acp deploy
   telnyx-acp eval --ci --out results.json
   
6. After the next change, compare with the last run:
   telnyx-acp eval --compare results.json
   

​

When a Conversation Goes Wrong

A user’s conversation is named by its id: the sender’s number for texts, the id the page keeps for the browser channel, or whatever was passed to --conversation. Phone calls are not traced. telnyx-acp traces <conversation> lists its newest runs; telnyx-acp traces <conversation> <run> shows one. A trace holds no message text, so reading one shows what the agent did, not what was said. logs has the agent’s own log lines for the same time.