Evaluation & Operations
Agent Tracing and Observability
What to record across prompts, model decisions, tool calls, state changes, approvals, and outcomes.
Capture the trajectory
A useful trace connects the user goal to model calls, tool requests, tool results, state transitions, and the final outcome. Correlation identifiers make a distributed task possible to reconstruct.
Protect sensitive data
Prompts and tool results may contain secrets or personal information. Redact at collection time, restrict trace access, and set retention rules that match the data's sensitivity.
Observe product outcomes
Token counts and tool latency explain system behavior, but business metrics show whether the agent helped. Track completion, correction, escalation, abandonment, and downstream error rates.