Skip to navigation
Simulation Suites

Generate a simulation suite

Generate and save a simulation suite from reviewed test requirements, and assign it to an assistant. First draft requirements with POST /eval/simulation/onboarding/requirements. The response includes the saved suite, its simulations, and requirement coverage, including any gaps.

Requests can take up to 3 minutes. Set a client timeout of at least 200 seconds.

Use a unique idempotencyKey for each suite generation request. Retrying with the same key does not create a duplicate suite:

  • On a timeout or a 5xx (502, 504, 524), retry with the same key. You get the saved suite, a 409 while it is still generating, or a fresh generation if the first one never saved.
  • On a 409 SIMULATION_ONBOARDING_GENERATION_IN_PROGRESS, the first request is still generating. Wait Retry-After seconds (15) and retry with the same key.
  • On a 409 SIMULATION_ONBOARDING_IDEMPOTENCY_KEY_REUSED, the key was used for a different assistant or requirements. Send a new key.

Review the generated simulations and coverage gaps. Revise the suite with PATCH /eval/simulation/onboarding/suite/{suiteId}, then run it with POST /eval/simulation/run.

Each organization can generate 20 suites per UTC day, shared with the dashboard, and 2 at a time. A retry that returns a saved suite does not count toward these limits. On a 429 SIMULATION_ONBOARDING_CONCURRENCY_LIMITED, wait Retry-After seconds (30) before retrying. On a 429 SIMULATION_ONBOARDING_QUOTA_EXCEEDED, wait until the daily limit resets at 00:00 UTC before retrying.

Authentication

AuthorizationBearer

Authenticate server-side requests with a private Vapi API key. Create or copy a key from the Vapi Dashboard and send it in the Authorization header as Bearer <token>. Keep private API keys out of client-side code and public repositories.

Request

This endpoint expects an object.
assistantIdstringRequiredformat: "uuid"
Assistant the suite tests and is assigned to.
idempotencyKeystringRequired1-200 characters
A unique key for this suite generation request. Retrying with the same key returns the saved suite, or a 409 while generation is in progress. Use a new key for a different assistant or requirements.
requirementslist of objectsRequired
The reviewed test requirements to generate simulations for, from the requirements draft. Each requirement must have a unique id.

Response

suiteobject
The generated simulation suite id and name.
assistantIdstring
Assistant the suite is assigned to.
simulationslist of objects
The suite's simulations, in order.
coverageobject
Counts of covered and total requirements, uncovered requirements, and scenarios available to retry.
requirementslist of objects

The test requirements used to generate the suite. Match their ids to coverage.gaps to identify uncovered requirements.

docsobject
Links to simulation testing best practices and tool mocking documentation.
reusedboolean

True when the request matched one already applied (same idempotencyKey or patchKey) and nothing changed.

failedEditSimulationIdslist of stringsOptional

Simulations an edit asked to rewrite that were not rewritten, for example because the rewrite did not generate in time. Retry them with a new patchKey.

toolGapslist of objectsOptional
Tool paths the suite's simulations do not exercise as configured, such as a tool mocked although it runs natively. Omitted when there are none.

Errors

400
Bad Request Error
403
Forbidden Error
404
Not Found Error
409
Conflict Error
422
Unprocessable Entity Error
429
Too Many Requests Error
502
Bad Gateway Error
503
Service Unavailable Error
504
Gateway Timeout Error