Manage simulations

Manage an existing simulation suite from the Dashboard or API. You can duplicate or edit its configuration, review and rerun results, cancel an active run, maintain test coverage, or delete the suite. If you have not created a suite, complete the Simulations quickstart first.

Edit a suite

Dashboard

cURL

Open Simulations, select Suites, and open the suite. Select Edit in the top-right corner, change its simulations, scenarios, personalities, or success criteria, then save.

Duplicate a suite

Duplicating a suite creates an independent copy of its simulations, targets, and notification settings. The duplicate uses the original suite name with (copy) appended, and later changes do not affect the original suite.

Dashboard

cURL

Open Simulations, select Suites, and open the suite. Open the suite actions menu, select Duplicate, then confirm. Vapi opens the duplicate after creating it.

Rename an AI tester

You cannot rename an AI tester from the Dashboard. An AI tester’s name comes from its personality, so rename the tester with the API by updating the personality with PATCH. Send only the name field.

curl -X PATCH "https://api.vapi.ai/eval/simulation/personality/<personality-id>" \
  -H "Authorization: Bearer $VAPI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "name": "New tester name" }'

Review run results

Dashboard

cURL

Open Simulations, then select Runs to see every run with its suite, iterations, assistant or squad, run date, and overall result (Passed or a failed count such as 1/1 failed). Filter by time range, status, or assistant or squad to find a run.

Open a run to see whether each simulation passed or failed, its evaluations, and its transcript. Open an active run to watch the transcript and evaluations update. If a run or iteration can’t complete, it shows a Failure reason describing what went wrong, such as no payment method on file. See the Simulations quickstart for the full walkthrough.

Rerun a suite

Dashboard

cURL

Open Simulations, select Runs, and find the completed run. Select Rerun, choose the Mode (chat or voice) and Iterations, then confirm. Each rerun creates a new run that you can compare with previous results.

Testing strategies

Build coverage gradually as the assistant or squad changes.

Smoke tests

Start with a core path, a direct intent, and one or two required Boolean evaluations. One iteration is useful while setting up or debugging the test, not as evidence of reliability. Repeat critical release checks based on risk and past variation. Chat mode gives quick feedback on conversation and tool logic.

Regression tests

Create a regression test when you fix a defect or find an unexpected response. Reproduce the original conditions in the scenario, give the simulation a descriptive name, and keep it in the suite that covers the affected behavior.

Edge-case tests

Test realistic variations such as an ambiguous request, an impatient customer, an unavailable appointment, a tool error, an interruption, or a handoff. Base AI tester personalities on the customer types the assistant or squad handles. Synthetic callers don’t reliably reproduce every interruption, silence, or background-noise condition. Keep controlled real-call checks for problems the simulation can’t reproduce.

Design evaluations

Keep each evaluation focused on one observable outcome. Use a descriptive name, choose a Boolean or numeric value that can be measured consistently, and avoid combining several independent requirements into one evaluation.

Choose voice or chat mode

Testing goal	Mode	Why
Iterate on prompts, tools, and conversation logic	Chat (`vapi.webchat`)	Runs without audio processing, so it is faster and costs less.
Investigate speech recognition, voice output, or interruptions	Voice (`vapi.websocket`)	Exercises synthetic audio. Review the recording, not just the pass/fail label.
Receive call-specific webhook data	Voice (`vapi.websocket`)	Start and end webhooks fire in both modes, but chat payloads omit call-specific fields.
Check representative voice journeys before launch	Voice (`vapi.websocket`)	Adds voice coverage. Also make controlled calls through the real phone path and sandbox integrations.

Review failures and sample passing runs. Compare automated judgments with human-reviewed examples, and listen to voice recordings when available. Follow run and maintain tests for release checks and failure triage.

Maintain simulation coverage

Update simulations when the behavior under test changes:

Change	Action
Prompt, tool, or configuration update	Rerun the affected suite. Change its evaluations only when the expected outcome changes.
New feature or conversation path	Add simulations for the core path and important failure paths.
Fixed defect	Add a regression simulation that reproduces the original failure.
New customer behavior or edge case	Add a representative scenario and AI tester personality.
Changed business requirement	Update the scenario intent and evaluations to describe the new expected outcome.