Simulations advanced

Advanced simulation options let you configure the AI tester, set variable values, mock tool responses, trigger lifecycle webhooks, and reuse structured outputs. Use them after you complete the Simulations quickstart and need more control over behavior or test conditions.

How it works

Advanced options belong to each simulation in a suite. They control how the AI tester behaves, which values the assistant or squad receives, what mocked tools return, and which lifecycle events send webhooks.

Open a suite and select Edit, then Next. The review step contains the Success criteria, Variables, and Tool mocks & webhooks tabs.

Configure the AI tester

The AI tester drives the simulated conversation, and its configuration lives in the personality. On a simulation’s Personality tab, expand Advanced settings to change how the AI tester reasons, listens, and speaks:

• Model: Choose the provider and model that control how the AI tester reasons. Use a provider and model you can access.
• Transcriber (optional): Override the AI tester’s speech-to-text provider, language, and model. If you leave it unset, the AI tester falls back to a built-in default.
• Voice: Choose the provider and voice for the AI tester. Select a configured voice or enter a custom voice ID.

Transcriber and voice apply to voice simulations. Chat simulations use the model only.

Mock tool responses

During a simulation, the assistant or squad under test runs its real tools. Mock a tool to return fixed text instead, so that tool never calls its live service or API. A mock intercepts only the tool it names; every other tool the assistant or squad calls still runs for real.

Dashboard

cURL

On the review step, open Tool mocks & webhooks. Under Tool mocks, select Add, then configure each mock with three fields:

• Tool: Select one of the tools configured on the assistant or squad. Only its configured tools, including custom tools, can be mocked.
• Mock result: Enter the string returned in place of the tool’s live output.
• Enabled: Toggle the mock on or off. A disabled mock is ignored, and the tool runs live.

The mock result is always passed to the assistant or squad as a string. In the Dashboard, enter plain text or JSON-formatted text directly. In a cURL JSON body, escape the quotation marks inside JSON-formatted text, as shown in the example. Use tool mocks to keep tool behavior deterministic, avoid side effects, and steer the conversation down a specific path without depending on a live service.

Common mock patterns

Choose a result that leads the assistant or squad through the behavior you want to test. These examples show how the result field appears in a cURL JSON body:

1
{
2
  "result": "{\"status\": \"success\", \"confirmationId\": \"APT-12345\"}"
3
}

These strings replace the selected tool’s output. They do not produce an HTTP error, add network latency, or cause a network timeout.

Send webhooks on run start and end

Turn on webhooks to notify your server when a simulation iteration starts or ends. Each webhook sends a POST request to the URL you configure.

Dashboard

cURL

On the Tool mocks & webhooks tab, under Webhooks, toggle Simulation start and Simulation end.

Vapi sends simulation.run.started when a simulation iteration begins. The example below is a voice payload. A chat payload omits the calls object:

1
{
2
  "type": "simulation.run.started",
3
  "simulationId": "8f2a9c1e-3b4d-4a7e-9c2f-1d6b0a5e7f34",
4
  "runId": "b7d4e2a0-6c9f-4e13-8a52-2f7c9b1d4e60",
5
  "simulationRunItemId": "c3e8f1a2-9d47-4b6c-8e30-5a1f2b9c7d84",
6
  "iterationNumber": 1,
7
  "isSimulation": true,
8
  "calls": {
9
    "testerCallId": "a1b2c3d4-...",
10
    "targetCallId": "e5f6a7b8-...",
11
    "listenUrl": "wss://..."
12
  }
13
}

Vapi sends simulation.run.ended when the simulation iteration finishes. The example below is a voice payload. A chat payload omits calls, endedReason, and recordingUrl:

1
{
2
  "type": "simulation.run.ended",
3
  "simulationId": "8f2a9c1e-3b4d-4a7e-9c2f-1d6b0a5e7f34",
4
  "runId": "b7d4e2a0-6c9f-4e13-8a52-2f7c9b1d4e60",
5
  "simulationRunItemId": "c3e8f1a2-9d47-4b6c-8e30-5a1f2b9c7d84",
6
  "iterationNumber": 1,
7
  "isSimulation": true,
8
  "calls": { "testerCallId": "a1b2c3d4-...", "targetCallId": "e5f6a7b8-...", "listenUrl": "wss://..." },
9
  "endedReason": "...",
10
  "canceled": false,
11
  "failureReason": "...",
12
  "startedAt": "2026-07-28T17:10:05.000Z",
13
  "endedAt": "2026-07-28T17:12:30.000Z",
14
  "transcript": "...",
15
  "messages": [],
16
  "recordingUrl": "https://..."
17
}

Both chat and voice payloads include simulationId, runId, simulationRunItemId, iterationNumber, and isSimulation. The ended event adds startedAt and endedAt, plus transcript and messages when you include them. The call-specific fields appear for voice simulations only: calls, endedReason, and recordingUrl. The calls object includes calls.listenUrl, a WebSocket URL for monitoring the run in real time. For the complete list of endedReason values, see Call ended reasons.

Reuse structured outputs

Each evaluation references a structured output. Reuse a structured output when several simulations should apply the same evaluation criterion. For more patterns, see structured output examples.

Set variable values for the assistant or squad

Variables provide values for the dynamic variables used by the assistant or squad during the simulation. On the Variables tab, add Name and Value pairs. Each name must match a {{variable}} placeholder in the assistant’s prompt, and the value is substituted during the run.

Use variables to test specific inputs, such as a customer name or account tier, without editing the assistant or squad.

Troubleshooting

Next steps