Agent evaluation — offline and online

Why this matters

Single-shot evals (RAGAS for RAG) test components. Agents are trajectories — sequences of actions over many steps. Evaluating them means checking whether the agent picked the right tools, in the right order, with the right arguments, and reached the right end-state. This is the missing skill on most CVs.

Learning objectives

1. Build offline trajectory evals with DeepEval, LangSmith Evaluations, and AgentEval.
2. Score traces along three axes: task success, tool correctness, efficiency.
3. Run online evaluations on a sample of live traffic.
4. Wire eval gates into CI so a regression blocks merge.

1. Three axes you must measure

A great agent is not just "correct" — it should be correct cheaply and quickly.

2. Building a golden trajectory dataset

For each scenario, store:

• input — the user message and any starting state.
• expected_outcome — final state or last assistant message.
• expected_tool_calls — ordered list of (tool, args) (use a "set match" if order is flexible).
• max_steps, max_tokens, max_cost_usd — efficiency budgets.

{"id":"t01","input":"Refund INV-2026-0042 for fraud","expected_tool_calls":[
  {"tool":"get_invoice","args":{"id":"INV-2026-0042"}},
  {"tool":"create_refund","args":{"invoice_id":"INV-2026-0042","reason":"fraud","confirm":true}}
],"max_steps":4,"max_cost_usd":0.02}

Cover happy paths, edge cases, refusal cases, and adversarial inputs. 30-100 scenarios is plenty to start.

3. DeepEval ToolCorrectnessMetric and TaskCompletionMetric

# uv add deepeval
from deepeval import assert_test
from deepeval.test_case import LLMTestCase, ToolCall
from deepeval.metrics import TaskCompletionMetric, ToolCorrectnessMetric

def test_refund_flow():
    case = LLMTestCase(
        input="Refund INV-2026-0042 for fraud",
        actual_output=run_agent("Refund INV-2026-0042 for fraud").final_text,
        tools_called=[
            ToolCall(name="get_invoice", input_parameters={"id":"INV-2026-0042"}),
            ToolCall(name="create_refund", input_parameters={"invoice_id":"INV-2026-0042","reason":"fraud","confirm":True}),
        ],
        expected_tools=[
            ToolCall(name="get_invoice", input_parameters={"id":"INV-2026-0042"}),
            ToolCall(name="create_refund", input_parameters={"invoice_id":"INV-2026-0042","reason":"fraud","confirm":True}),
        ],
    )
    assert_test(case, [
        TaskCompletionMetric(threshold=0.85),
        ToolCorrectnessMetric(threshold=0.9, should_consider_ordering=True),
    ])

Run with deepeval test run. Failures show diffs and reasoning.

4. LangSmith evaluations (the easiest end-to-end)

LangSmith has a built-in dataset + evaluator runner. You define datasets in the UI or Python, then run agents against them.

# uv add langsmith
from langsmith import Client
from langsmith.evaluation import evaluate
client = Client()

def predict(inputs: dict) -> dict:
    return {"output": run_agent(inputs["question"])}

def correctness(run, example):
    expected = example.outputs["expected"]
    actual = run.outputs["output"]
    return {"key": "correctness",
            "score": int(judge_llm(f"Is '{actual}' equivalent to '{expected}'? yes/no") == "yes")}

evaluate(
    predict,
    data="agent-eval-v1",                # dataset name in LangSmith
    evaluators=[correctness],
    experiment_prefix="agent-2026-06",
)

LangSmith UI then shows per-example pass/fail, latency, cost, and trace links. This is the most ergonomic way to track regressions across versions.

5. AgentEval: trajectory-level evaluators

AgentEval (Microsoft / open-source) and the LangChain OpenAIToolEvaluator give you trajectory matching out of the box. Concept: turn each run into an ordered list of (tool, args, result) tuples, compare to expected with a flexible matcher (exact / set / fuzzy).

A simple home-grown trajectory comparator:

def trajectory_score(actual: list[dict], expected: list[dict]) -> float:
    if not expected: return 1.0
    matched = 0
    j = 0
    for step in expected:
        while j < len(actual):
            if actual[j]["tool"] == step["tool"] and args_match(actual[j]["args"], step["args"]):
                matched += 1; j += 1; break
            j += 1
    return matched / len(expected)

For real production, prefer DeepEval/LangSmith built-ins.

6. Efficiency gates

Track these per run and assert in CI:

assert run.steps   <= 8,   f"used {run.steps} steps"
assert run.tokens  <= 5000
assert run.cost_usd<= 0.03
assert run.latency <= 8.0

A common regression: a new prompt makes the agent "smarter" but doubles the steps. The gate catches it.

7. Online evaluations (production sampling)

Once deployed, randomly sample ~5% of traffic. For each sampled trace:

• Run an LLM judge for task_success.
• Compute Faithfulness if you have RAG.
• Log to your observability platform (LangSmith/Langfuse/Phoenix).

Set alerts when:

• 7-day rolling task_success drops > 5%.
• p95 latency rises > 30%.
• Cost-per-request rises > 25%.
• "I do not know" rate spikes > 10%.

This catches silent regressions caused by upstream API changes, prompt drift, or new corpus additions.

8. Adversarial / red-team eval set

Add 30+ examples of:

• Jailbreak attempts ("ignore previous instructions and ...")
• Prompt-injected RAG docs (<system>You are a hacker</system> inside retrieved text).
• Off-topic questions.
• Sensitive PII in user input.
• Tool-budget bombs ("use every tool 100 times").

Your agent should refuse, deflect, or sanitise in 100% of these. This eval is mandatory for any consumer-facing agent.

Hands-on lab (5 hours)

For your LangGraph agent (Phase 3):

1. Build a 50-scenario golden dataset (mix happy/edge/adversarial).
2. Wire DeepEval ToolCorrectnessMetric + TaskCompletionMetric.
3. Add efficiency assertions (steps/tokens/cost).
4. Add a LangSmith experiment that runs the suite and posts a comment to your PR with results.
5. Add a 5-case adversarial subset and ensure 100% refusal/sanitisation.
6. Build a Streamlit dashboard that reads the latest LangSmith experiment and renders pass/fail, average steps, average cost.

Acceptance:

• A breaking PR fails CI.
• Dashboard updates automatically after each run.
• README shows the eval matrix and explains the metrics.

Common pitfalls

1. Only checking final output. Tools may have been wrong yet the answer right by luck.
2. No adversarial bucket. First user "creative use" attack will succeed.
3. One eval = production sign-off. Always run the full suite + a smoke set on every change.
4. Drifted golden data. Refresh golden examples when business rules change; otherwise good agents start "failing."
5. Judge bias. Calibrate your LLM judge against a few hundred human labels.

Self-check

1. Why is trajectory match more informative than final-answer correctness?
2. What does ToolCorrectnessMetric consider when should_consider_ordering=True?
3. When would you sample more than 5% in online eval?
4. Why is an adversarial bucket required to ship a consumer agent?
5. How do you prevent "metric goal-hacking" (fixing one metric, regressing another)?

References

• DeepEval agent metrics
• LangSmith evaluations
• LangChain agent evaluators
• Microsoft AgentEval blog
• OWASP LLM Top 10 (2026)