Creates a scenario, the AI tester's intent plus the success criteria that score a run.
Request
This endpoint expects an object.
namestringRequired<=80 characters
The display name of the scenario, for example Book an appointment.
instructionsstringRequired<=10000 characters
What the AI tester should try to accomplish in the conversation. Write it as the AI tester’s goal, for example Book an appointment for next week and confirm the time.
evaluationslist of objectsRequired
The checks that decide whether a run passes. Each evaluation compares a structured output against an expected value. At least one evaluation is required to run.
hookslist of objectsOptional
Hooks to run on simulation lifecycle events
targetOverridesobjectOptional
Overrides to inject into the simulated target assistant or squad
toolMockslist of objectsOptional
Mock results for the assistant or squad's tools during the simulation, so the run stays deterministic without calling real services.
pathstring or nullOptionalformat: "/^[a-zA-Z0-9][a-zA-Z0-9._-]*(?:\/[a-zA-Z0-9][a-zA-Z0-9._-]*){0,2}$/"<=255 characters
Optional folder path for organizing scenarios.
Supports up to 3 levels (e.g., “dept/feature/variant”).
Maps to GitOps resource folder structure.
Response
idstringformat: "uuid"
This is the unique identifier for the scenario.
orgIdstringformat: "uuid"
This is the unique identifier for the organization this scenario belongs to.
createdAtdatetime
This is the ISO 8601 date-time string of when the scenario was created.
updatedAtdatetime
This is the ISO 8601 date-time string of when the scenario was last updated.
namestring<=80 characters
This is the name of the scenario.
instructionsstring<=10000 characters
This is the script/instructions for the tester to follow during the simulation.
evaluationslist of objects
This is the structured output-based evaluation plan for the simulation.
Each item defines a structured output to extract and evaluate against an expected value.
hookslist of objectsOptional
Hooks to run on simulation lifecycle events
targetOverridesobjectOptional
Overrides to inject into the simulated target assistant or squad
toolMockslist of objectsOptional
Scenario-level tool call mocks to use during simulations.
pathstring or nullOptionalformat: "/^[a-zA-Z0-9][a-zA-Z0-9._-]*(?:\/[a-zA-Z0-9][a-zA-Z0-9._-]*){0,2}$/"<=255 characters
Optional folder path for organizing scenarios.
Supports up to 3 levels (e.g., “dept/feature/variant”).
Maps to GitOps resource folder structure.
