> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.vapi.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.vapi.ai/_mcp/server.

# Plan test coverage

> Prioritize test coverage by risk, mapping customer outcomes, critical decisions, failure paths, guardrails, and voice behavior to Evals and Simulations.

## Overview

Good coverage does not mean scripting every sentence a caller might say. It means having enough evidence that your agent handles the goals, decisions, and risks that matter to your business.

Start with a focused set of high-value tests. Add coverage as you learn from manual testing, support issues, and production conversations.

## Start with clear requirements

Write down what the caller should be able to accomplish and what the agent must never do. Use language that a support operator, product manager, and subject-matter expert can judge the same way.

| Too vague                         | Clear enough to test                                                                          |
| --------------------------------- | --------------------------------------------------------------------------------------------- |
| The agent is helpful.             | If the requested time is unavailable, the agent offers available alternatives.                |
| The agent protects customer data. | The agent does not share appointment details until the caller verifies their identity.        |
| The agent handles transfers well. | The agent transfers billing disputes to Billing and explains the handoff before transferring. |

> **Check**
>
> A requirement is ready to test when two informed teammates would independently reach the same pass or fail decision.

## Prioritize by risk

You do not need hundreds of tests to get started. Test these areas first:

* **High impact:** A failure could expose private data, move money, make an unauthorized change, miss an urgent escalation, or create legal risk.
* **High frequency:** Callers use the behavior often or depend on it to complete the agent's main job.
* **High uncertainty:** The behavior is new, recently changed, inconsistent, or dependent on an external tool.

Cover high-impact behavior even when it is rare. Then cover the most common customer goals and the areas that change most often.

## Build coverage in layers

Use this checklist for each important customer goal.

* [ ] **Core outcome:** The caller completes the goal and receives an accurate confirmation or next step. Start with a Simulation.
* [ ] **Critical decisions:** The agent asks required questions and uses the right tools and information. Start with Evals.
* [ ] **Both directions:** The agent acts when it should and waits or refuses when it should not. Start with Evals.
* [ ] **Missing or unclear information:** The agent asks for what it needs instead of guessing. Start with an Eval, then a Simulation.
* [ ] **Tool and service failures:** The agent explains the problem, avoids a false confirmation, and offers a safe next step. Start with an Eval, then a Simulation.
* [ ] **Guardrails:** The agent protects private data, follows policy, and escalates when required. Use Evals and Simulations.
* [ ] **Caller variation:** Vague, impatient, confused, or changing requests still receive appropriate help. Use Simulations.
* [ ] **Handoffs:** The agent chooses the right destination, gives context, and handles an unavailable destination. Use Evals and Simulations.
* [ ] **Voice experience:** Names, dates, numbers, and domain terms are understood and spoken clearly. Review voice Simulation recordings for pacing and turn-taking.
* [ ] **Known failures:** Each important fixed issue has a test that recreates its conditions. Use whichever tool reproduces it best.

Many requirements need both tools. Use an Eval to protect a critical choice and a Simulation to check that the full conversation still succeeds.

## Cover both success and restraint

Testing only the action you want can create a new problem: an agent that takes that action too often. Pair positive cases with negative cases.

For an identity check, test that the agent:

* Shares account details after successful verification
* Does not share account details after failed verification
* Asks for missing verification information
* Escalates or ends safely after repeated failed attempts

Use the same pattern for refunds, discounts, appointments, transfers, cancellations, and other consequential actions.

## Example coverage map

For an appointment scheduling agent, a useful starting map might look like this:

| Requirement                 | Decision coverage with Evals                                   | Outcome coverage with Simulations                                              |
| --------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| Book an available time      | Use the booking tool with the correct date, time, and customer | The booking tool reports success and the caller hears the correct confirmation |
| Protect appointment details | Share details only after successful verification               | An unverified caller cannot obtain private appointment information             |
| Handle an unavailable time  | Do not claim success; ask the caller to choose an alternative  | The caller chooses an available alternative and completes the booking          |
| Recover from a tool failure | Do not invent a confirmation number                            | The caller receives an honest explanation and a safe next step                 |
| Support changes of mind     | Choose the new action instead of continuing the old one        | A caller can switch from booking to canceling without an unintended booking    |

This map describes behavior, not exact wording. An agent should not fail because it used a different natural phrase or took a different valid route.

## Set a launch bar

Before a voice agent handles production traffic, confirm that:

* Every primary customer goal has reusable end-to-end Simulation coverage
* Every high-risk decision has cases for when the action should and should not occur
* Each primary goal includes at least one likely failure or recovery case
* Critical handoffs and guardrails have dedicated coverage
* Primary and high-risk outcomes pass repeatedly, with the number of runs based on the impact and variability of the behavior
* Critical conversations have been tested in voice mode
* A person has reviewed representative transcripts and recordings
* Tools that can make real changes use safe test data or [mock responses](/observability/simulations-advanced#mock-tool-responses)

Treat this as a minimum, not a finish line. Increase coverage for regulated, high-volume, or high-consequence use cases.

## Keep coverage current

Test coverage is a living record of what your team expects. Update it when:

* A customer or support issue reveals a new failure mode
* You add a customer goal, tool, handoff, language, or policy
* You change a prompt, model, voice, transcriber, or provider
* A business requirement changes

When a test fails, first decide whether the agent is wrong or the expectation has changed. Do not weaken a test only to make it pass. Review the transcript, correct the agent or the requirement, and keep the test as regression coverage.

## Next steps

#### [Testing voice agents](/test/voice-testing)

Review how Evals and Simulations work together.

#### [Test outcomes with Simulations](/test/simulations-best-practices)

Turn customer goals into complete conversation tests.

#### [Test decisions with Evals](/test/evals-best-practices)

Create focused checks for critical choices and guardrails.

#### [Run and maintain tests](/test/run-and-maintain-tests)

Run the right tests before release and learn from production failures.