Evaluation and Observability

Agents are hard to evaluate: outputs are open-ended, processes are non-deterministic, and workflows involve multiple steps. But without evaluation, iteration is impossible—evaluation (offline quality measurement) and observability (online runtime visibility) are complementary; only together can they transform an agent from "looks okay" to "measurable and improvable."

Overview

Agents are hard to evaluate for three reasons: outputs are open-ended (no single correct answer), processes are non-deterministic (different results for the same prompt), and workflows are multi-step (a single task may involve twenty tool calls). But without evaluation, you cannot iterate—when you change the prompt, switch models, or add tools, how do you know it "got better" rather than "broke differently"? This article covers two complementary practices: evaluation (eval)—offline, repeatable quality measurement; and observability—online visibility into what the agent is actually doing at runtime. The former answers "did it improve overall?", while the latter answers "which step went wrong?" Integrating both into your iteration loop allows agents to move from "looks okay" to "measurable and improvable."

Evaluation: Offline Quality Measurement

flowchart LR
    D["eval set<br/>representative tasks + expectations/rubrics"] --> R["Run current version"]
    R --> J["Score"]
    J -->|Programmatic?| P["Code assertions"]
    J -->|Open-ended?| L["LLM-as-judge"]
    J -->|Has completion criteria?| O["Outcome scoring"]
    P --> I["Aggregate & Compare"]
    L --> I
    O --> I
    I -->|Modify prompt/model/tools| R

Use scoring methods in order of reliability (highest to lowest): prefer the former over the latter whenever possible.

1. Programmatic Checks (Most Reliable)

If it can be determined by code, never delegate it to a model. "Return JSON passing schema validation," "CSV has a numeric column price," "tests pass," "contains specific keyword fields"—use direct assertions. Deterministic, zero cost, no bias. Structured outputs (output_config.format / strict: true) make these checks stable and reliable (see Prompt Engineering and Structured Outputs).

2. LLM-as-Judge (Use Only for Open-Ended Tasks)

For things like summary quality or relevance that cannot be judged programmatically, use one model as the judge. The key is reducing bias:

• Provide clear rubrics: Break down "good or bad" into independently verifiable criteria ("did it cite sources?", "did it answer all three sub-questions?"), rather than letting the judge give a holistic score based on gut feeling. Vague rubrics → noisy scores.
• Separate judge from subject: Don't let the same model grade its own output (self-endorsement). Use different models, or at least independent contexts, as judges.
• Beware of known biases: Judges tend to favor longer answers, prefer their own model's style, and are influenced by option order—more specific rubrics mitigate these biases.
• Evaluate the scoring itself: Sample and manually verify the judge's scores to confirm reliability before scaling up.

3. Outcome-Based (When Completion Criteria Exist)

Managed Agents make "result-oriented evaluation" a primitive. Send a user.define_outcome event with:

• description (task), rubric ({type:"text"|"file"}, required, write independently verifiable criteria), max_iterations (default 3, max 20).

The platform runs an independent grader in an iterate → grade → revise loop, emitting span.outcome_evaluation_start/ongoing/end events each round. end.result ∈ {satisfied, needs_revision, max_iterations_reached, failed, interrupted}, continuing until criteria are met or the limit is reached. This builds "evaluation + automatic improvement" directly into the runtime. Suitable for tasks with clear "what completion looks like" (reports, models, documents).

4. Regression

Every time you encounter a failure, solidify it into an eval case. This is how your eval set grows stronger over time—every online failure adds another offline defense line.

Most important practice: Start with a small, real eval set before tuning. Even just 20 representative tasks (including known failures/edge cases, not just happy paths) is far better than guessing based on a single instance.

Observability: Online Runtime Visibility

Once an agent is live, you need to answer: What steps did it take? Which step was slow/failed? How many tokens were consumed? Why did it stop?

• Trace / Span: Break a single run into visible steps—each tool_use/tool_result/thinking/model call is a span. Managed Agents provide event streams directly; span.model_request_end includes model_usage (token details for this inference), enabling step-by-step attribution of latency and cost.

• Usage (Cost + Cache Health): Each response's usage provides:

  Field	What to look for
  input_tokens	Uncached input (full price)
  output_tokens	Output
  cache_read_input_tokens	Cache hits (~0.1x); zero over time = silent invalidator (see Context Engineering)
  cache_creation_input_tokens	Cache writes (~1.25x)

  Note: input_tokens is only the uncached remainder; total = sum of all three—don't look at just one field and assume no tokens were spent.

• Stop Reason Distribution: Proportions of end_turn / tool_use / max_tokens / refusal form a health dashboard. High max_tokens = output truncated (increase limit or use streaming); abnormal rise in refusal = prompt triggered refusal; many pause_turn = server-side tool loops continuing.

• OpenTelemetry: Export traces/metrics to standard backends for unified viewing.

Real example: Our site's static generator initializes OTel during build—the log line OTel tracer initialised, exporting to ... confirms this. The same "trace every step, quantify every instance" approach applies directly to LLM applications.

Integrating Both into the Loop

Observability answers "which step went wrong"; evaluation answers "overall did it improve?" Without the former, you don't know why it failed; without the latter, you don't know if changes are real improvements. Missing either makes iteration blind guessing.

Best Practices

• Start with a small, real eval set before tuning. Even 20 representative tasks (including known failures/edge cases) is far better than guessing based on a single instance.
• Use scoring methods in order of reliability. Programmatic checks > LLM-as-judge > outcome scoring; never delegate what can be asserted to a model (Structured Outputs make these checks stable).
• Separate judge from subject, provide independently verifiable rubrics. No self-endorsement; break down "good or bad" into itemized, verifiable criteria to reduce noise and bias.
• Solidify every failure into a regression eval. Every online failure becomes an offline defense line, strengthening the eval set over time.
• Trace every step, quantify every instance. Break runs into spans; attribute latency and cost step-by-step via usage/stop_reason; export to standard backends via OTel for unified viewing.
• Monitor cache hit rates. cache_read_input_tokens consistently zero = silent invalidator, causing costs to quietly double.
• Look at aggregates for non-deterministic systems, don't be misled by single-instance feelings. Pass rates, token distributions, stop_reason proportions; attribute batch results by custom_id/trace id, never by order.

Trade-offs and Failure Modes

• Only testing happy paths: Crashes on edge cases in production → eval sets must include failures/edge cases and grow with incidents (regression).
• Judge = Subject: Self-endorsement → Separate judge from subject; corresponds to agent engineering's "authoring and review in separate lanes" (see Multi-Agent Orchestration).
• Vague rubrics: "Looks good" cannot be stably scored → Write independently verifiable itemized criteria.
• Looking at single instances instead of distributions: Non-deterministic systems require aggregate views (pass rates, token distributions, stop_reason proportions); don't be misled by single-instance feelings.
• No attribution: Batch/parallel results don't match specific instances → Use custom_id / trace id for attribution, never by order (Batches results are inherently unordered).
• Not monitoring cache hit rates: Costs quietly multiply before discovery → Add cache_read_input_tokens to monitoring dashboards.

References

• Anthropic Official Documentation: Building Evals, Managed Agents Define Outcomes & Observability, Handling Stop Reasons (platform.claude.com)

Keywords: eval, eval set, programmatic check, structured outputs, LLM-as-judge, rubric, judge bias, outcome-based, user.define_outcome, grader, span.outcome_evaluation, max_iterations, regression, observability, trace, span, model_usage, usage, input_tokens, output_tokens, cache_read_input_tokens, stop_reason distribution, OpenTelemetry, OTel, authoring vs review lane, custom_id