Track · For qa engineers
Agent QA & Testing
How to test non-deterministic agents: evalsets, criteria, LLM judges, simulated users, traces, CI and production monitoring.
Module 1 Testing mindset
Module 2 ADK evaluation
- 03 How ADK evaluates an agent Evalsets, trajectory and response scoring, pass and fail, animated. Start →
- 04 Test files vs evalsets .test.json for unit-style, .evalset.json for integration. Start →
- 05 Trajectory and response criteria tool_trajectory_avg_score and response_match_score. Start →
- 06 LLM-as-judge criteria final_response_match_v2, rubrics, hallucinations, safety. Start →
- 07 Multi-turn and user simulation Letting an AI play the user. Start →
- 08 Environment simulation Testing without real backends. Start →
- 09 Custom metrics Scoring what matters to your product. Start →
- 10 Efficiency metrics Tool calls, model calls, tokens and latency. Start →
- 11 Running evals: web UI, CLI and pytest adk web, adk eval and AgentEvaluator. Start →
- 12 Evals in CI/CD Blocking a merge when quality drops. Start →
- 13 Testing deployed agents Smoke tests against a live endpoint. Start →
- 14 Optimization Improving an agent from eval results. Start →
Module 3 Agent Platform evaluation and observability
- 15 Agent evaluation on Agent Platform Metrics, datasets and runs. Start →
- 16 Offline and simulated evaluation Testing before release. Start →
- 17 Online monitors and quality alerts Evaluating live traffic continuously. Start →
- 18 Reading agent traces Debugging failures with OpenTelemetry traces. Start →
Module 4 CX Agent Studio evaluation
- 19 Golden, scenario and simulation tests The three test types, run side by side. Start →
- 20 Expectations and metrics Semantic match, tool correctness, goal satisfaction. Start →
- 21 Personas and the simulator Testing like real callers. Start →
- 22 Batch upload, export and versions Managing test suites at scale. Start →
- 23 AI issue finding and hill climbing Improving an agent step by step. Start →