Simulations quickstart

In about 10 minutes, you’ll run an AI tester through an appointment-booking conversation with Riley and get a pass or fail result. You’ll also have a reusable simulation suite that you can run again after any change.

What you’ll build

A reusable simulation suite that asks an AI tester to book an appointment with Riley, Vapi’s appointment-scheduler template assistant. A Boolean evaluation checks whether Riley books the appointment for next Monday at 2:00 PM.

Prerequisites

A Vapi account.

You create Riley and the structured output during this quickstart.

Build and run a simulation

Dashboard

cURL

Create Riley from the appointment-scheduler template

In the Vapi Dashboard, go to Assistants. Open the arrow next to Create Assistant, then select Appointment Scheduler. Vapi creates Riley immediately.

Create a simulation suite

Open Simulations, then select Suites. Select Create suite, give the suite a name, add Riley under Assistants or squads, then select Next.

A suite opens in view mode. To change it, open the suite and select Edit in the top-right corner.

Create a new simulation suite dialog with a suite name and Riley selected under Assistants or squads

Configure the scenario

The suite opens with its first simulation. In the Scenario tab, enter Book an appointment as the Scenario name. Paste the following text into Intent:

You are calling to book an appointment for next Monday at 2pm.
Confirm your identity when asked and provide any required information.
End the call once you receive a confirmation number.

A suite can contain multiple simulations, each with its own scenario and personality. Select Add simulation in the suite editor to add another simulation.

Scenario tab with the Book an appointment scenario name and appointment-booking intent

Choose a personality

Open the Personality tab and select a built-in personality marked (Default). Keep its behavior and advanced settings unchanged for this first simulation.

Personality tab with the Skeptical Sam default personality selected

Add success criteria

Select Next. On the Success criteria tab, select Create structured output and configure this structured output:

Setting	Value
Name	`appointment_booked`
Type	Boolean
Description	Return `true` if Riley books the appointment for next Monday at 2:00 PM.
Comparator	`equals (=)`
Expected value	`true`

You must add at least one structured output before you can run a simulation.

Save and run

Select Save & run. In Run suite, confirm Riley, choose a mode, and set Iterations to 1:

Chat

Runs faster and costs less because it skips audio evaluation.

Voice

Tests the complete audio experience at about twice the cost.

Select Confirm to start the run.

Run suite dialog with Riley selected, Chat mode selected, and Iterations set to 1

Review results

Wait for the run to move through queued, running, and ended, then select View results to see what passed or failed.

To find the run later, open Simulations, then select Runs. Your run appears with its Status and Simulation result, along with the suite name, iterations, Riley, and run date. Filter by time range, status, or assistant or squad, then select the suite name to open the results.

The header shows Riley, the mode, run date, and overall result (Passed or a count such as 1/1 failed). The list on the left shows one result for each simulation and iteration. Select a result to review it:

The Success criteria panel appears next to the Transcript. The appointment_booked evaluation passes when Riley books the appointment for next Monday at 2:00 PM. The transcript lists the conversation turn by turn between the AI tester and Riley.

Passed simulation run with Success criteria beside the Transcript

Select Rerun to run the suite again.

Next steps

Simulations advanced

Configure AI testers, variables, tool mocks, webhooks, and structured outputs.

Simulations overview

What Simulations are and when to use them over Evals.

Structured outputs quickstart

Define the structured outputs your evaluations check against.

Manage simulations

Edit suites, review and rerun results, and maintain test coverage.