Manage simulations

Manage an existing simulation suite from the Dashboard or API. You can edit its configuration, review and rerun results, cancel an active run, maintain test coverage, or delete the suite. If you have not created a suite, complete the Simulations quickstart first.

Edit a suite

Dashboard

cURL

Open Simulations, select Suites, and open the suite. Select Edit in the top-right corner, change its simulations, scenarios, personalities, or success criteria, then save.

Rename an AI tester

You cannot rename an AI tester from the Dashboard. An AI tester’s name comes from its personality, so rename the tester with the API by updating the personality with PATCH. Send only the name field.

$ curl -X PATCH "https://api.vapi.ai/eval/simulation/personality/<personality-id>" \
>   -H "Authorization: Bearer $VAPI_API_KEY" \
>   -H "Content-Type: application/json" \
>   -d '{ "name": "New tester name" }'

Review run results

Dashboard

cURL

Open Simulations, then select Runs to see every run with its suite, iterations, assistant or squad, run date, and overall result (Passed or a failed count such as 1/1 failed). Filter by time range, status, or assistant or squad to find a run.

Open a run to see whether each simulation passed or failed, its evaluations, and its transcript. Open an active run to watch the transcript and evaluations update. If a run or iteration can’t complete — for example, no payment method on file — it shows a Failure reason describing what went wrong. See the Simulations quickstart for the full walkthrough.

Rerun a suite

Dashboard

cURL

Open Simulations, select Runs, and find the completed run. Select Rerun, choose the Mode (chat or voice) and Iterations, then confirm. Each rerun creates a new run that you can compare with previous results.

Testing strategies

Build coverage gradually as the assistant or squad changes.

Smoke tests

Start with the core path that must always work. Use a direct intent, one or two required Boolean evaluations, and a single iteration. Chat mode is useful for validating conversation and tool logic quickly.

Regression tests

Create a regression test when you fix a defect or find an unexpected response. Reproduce the original conditions in the scenario, give the simulation a descriptive name, and keep it in the suite that covers the affected behavior.

Edge-case tests

Test realistic variations such as an ambiguous request, an impatient customer, an unavailable appointment, a tool error, an interruption, or a handoff. Base AI tester personalities on the customer types the assistant or squad handles.

Design evaluations

Keep each evaluation focused on one observable outcome. Use a descriptive name, choose a Boolean or numeric value that can be measured consistently, and avoid combining several independent requirements into one evaluation.

Choose voice or chat mode

Testing goal	Mode	Why
Iterate on prompts, tools, and conversation logic	Chat (`vapi.webchat`)	Runs without audio processing, so it is faster and costs less.
Validate speech recognition, voice output, or interruptions	Voice (`vapi.websocket`)	Exercises the complete audio path.
Receive call-specific webhook data	Voice (`vapi.websocket`)	Start and end webhooks fire in both modes, but chat payloads omit call-specific fields.
Perform final pre-launch validation	Voice (`vapi.websocket`)	Tests the assistant or squad in an end-to-end voice conversation.

Maintain simulation coverage

Update simulations when the behavior under test changes:

Change	Action
Prompt, tool, or configuration update	Rerun the affected suite. Change its evaluations only when the expected outcome changes.
New feature or conversation path	Add simulations for the core path and important failure paths.
Fixed defect	Add a regression simulation that reproduces the original failure.
New customer behavior or edge case	Add a representative scenario and AI tester personality.
Changed business requirement	Update the scenario intent and evaluations to describe the new expected outcome.