ObservabilityMonitoring

Monitoring quickstart

Set up automated quality monitoring for your voice AI agents

Overview

Monitoring lets you automatically track quality and detect issues across your voice AI agents. Instead of manually reviewing calls, you define monitors that continuously evaluate your call data against thresholds and alert you when something goes wrong.

What is monitoring?

Monitoring is Vapi’s automated quality and effectiveness system for voice AI. You create monitors that periodically evaluate call data using analytics queries (Insights), compare results against thresholds you define, and generate issues when those thresholds are exceeded. Your team receives alerts through notifiers, which are alert channels such as email, Slack, or webhooks, so you can investigate and resolve problems quickly.

Core concepts

  • Monitors define what to watch, which assistants to target, and when to evaluate
  • Triggers run on a schedule and evaluate call data against thresholds
  • Issues are created when a threshold is exceeded and a trigger fires, tracking the problem from detection to resolution
  • Notifiers are alert channels (email, Slack, or webhook) that deliver notifications to your team when issues are created

Monitor categories

  • Technical tracks system-level failures like API errors, provider outages, and timeouts
  • Infrastructure tracks resource utilization, latency, and capacity issues
  • Effectiveness tracks assistant performance metrics like task completion and user satisfaction
  • Compliance tracks regulatory and policy adherence across conversations

What you’ll build

In this quickstart, you will create a monitor that tracks error rates across your assistants and alerts you when the number of errors exceeds a threshold. You will then view an issue, analyze its root cause, and resolve it.

How it works

1

Create a monitor

Define a monitor targeting specific assistants or all assistants. Choose a category and set the monitor to active.

2

Define triggers

Configure triggers with schedules or intervals, thresholds, and severity levels. Each trigger references an Insight that defines the analytics query to run.

3

Triggers evaluate call data

On each scheduled interval, the trigger runs its Insight query against your call data and compares the result to the threshold you defined.

4

Issues are created

When a threshold is exceeded and a trigger fires, an issue is created with details about the affected calls, the trigger that fired, and the evaluation window.

5

Notifiers alert your team

If notifiers are configured on the trigger, your team receives notifications via email, Slack, or webhook with issue details.

6

Investigate and resolve

Review the issue, run AI-powered root cause analysis, acknowledge the issue, fix the underlying problem, and mark it as resolved.

Prerequisites

Vapi account

Sign up at dashboard.vapi.ai

API key

Get your API key from API Keys in the sidebar

You need existing assistants with call data for monitoring to detect issues. Monitors evaluate historical call data, so triggers will not fire until your assistants have processed calls.

Step 1: Set up notifiers

Notifiers are alert channels (email, Slack, or webhook) that send notifications when issues are detected. You configure them as credentials in the Dashboard.

2

Configure the notifier

  1. Choose a notifier type: Email, Slack, or Webhook
  2. For email: enter the recipient email address
  3. For Slack: paste your Slack webhook URL
  4. For webhook: enter your endpoint URL
  5. Give the notifier a name (e.g., “Engineering Slack Channel”)
  6. Click Save
3

Copy the credential ID

After saving, copy the credential ID from the notifier details. You will need this when configuring alerts on your monitor triggers.

You can create multiple notifiers to send alerts to different channels depending on the severity of the issue.

Step 2: Create a monitor

Define a monitor that tracks error rates and alerts you when errors exceed a threshold.

2

Configure basic settings

  1. Name: Enter “Error Rate Monitor”
  2. Description: Add “Tracks API and provider errors across all assistants”
  3. Category: Select Technical
3

Choose target assistants

  1. Select All Assistants to monitor every assistant in your organization
  2. Alternatively, select Specific Assistants and choose individual assistants from the dropdown
4

Configure escalation thresholds

  1. Set Severity to Error
  2. Set the Comparator to Greater than
  3. Set the Value to 5 (triggers when more than 5 errors are detected)
  4. Set Check frequency to every 1 hour
5

Enable notifiers

  1. Toggle Notifiers to enabled
  2. Select one or more notifiers from the dropdown
  3. Click Save Monitor

The insightId references an Insight, which is an analytics query that defines what data to evaluate. When you configure a monitor’s escalation thresholds in the Dashboard, the Insight is created automatically. When using the API, you need to create the Insight first and reference its ID here.

Managing monitors

View, edit, and delete all your monitors from the Monitors page in the Dashboard sidebar (under Observe).

Via the API:

  • GET /monitoring/monitor — list all monitors
  • PATCH /monitoring/monitor/:id — update a monitor’s targets, triggers, or thresholds
  • DELETE /monitoring/monitor/:id — remove a monitor

Targeting assistants

Setting "targets": "*" monitors all current and future assistants in your organization. Any assistant created after the monitor is set up is automatically included.

To monitor only specific assistants, pass an array of assistant IDs using the targets array. The id field is the assistant ID — the UUID you see in the Dashboard or get from GET /assistant.

Specific assistant targets
1{
2  "targets": [
3    {
4      "type": "assistant",
5      "id": "c3d4e5f6-a7b8-9012-cdef-234567890abc"
6    },
7    {
8      "type": "assistant",
9      "id": "d4e5f6a7-b8c9-0123-defa-345678901bcd"
10    }
11  ]
12}

Schedule-based triggers

Instead of an interval, you can use a calendar schedule for more precise control:

Schedule-based trigger
1{
2  "insightId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
3  "schedule": {
4    "minute": [0],
5    "hour": [9, 17],
6    "dayOfWeek": [1, 2, 3, 4, 5]
7  },
8  "threshold": {
9    "type": "number",
10    "comparator": "gt",
11    "value": 10
12  },
13  "severity": "warning"
14}

This trigger evaluates at 9:00 AM and 5:00 PM on weekdays.

Step 3: View issues

When a threshold is exceeded and a trigger fires, an issue is created. You can view and manage issues in the Dashboard or via the API.

2

Browse and filter issues

  1. Browse the issues table showing monitor name, severity, status, and timestamps
  2. Filter by status: New, In Progress, or Resolved
  3. Sort by severity or creation date
3

View issue details

  1. Click an issue to open the detail panel
  2. Review the trigger that fired, the threshold that was exceeded, and the evaluation window
  3. View the list of affected calls

Step 4: Analyze an issue

Use AI-powered root cause analysis to understand why an issue occurred and get actionable suggestions for fixing it.

1

Open the issue

  1. Navigate to Issues in the sidebar
  2. Click on the issue you want to analyze
2

Run analysis

  1. Click the Analyze button in the issue detail panel
  2. Wait for the AI-powered analysis to complete
  3. Review the summary, root cause, impact assessment, and suggestions

Analysis results are cached for 1 hour. Subsequent requests within that window return the cached result immediately.

Use the callId values from the issue’s calls array to review specific call logs and recordings for deeper investigation. Each call ID links directly to the call details in your Dashboard.

Step 5: Resolve an issue

After investigating and fixing the underlying problem, acknowledge and resolve the issue to track your team’s response. Resolve an issue once you’ve deployed a fix and confirmed the problem no longer recurs — this signals to your team that the root cause has been addressed. Acknowledgment and resolution timestamps help measure your team’s incident response times.

Issues are a single shared resource. Status changes made in the Dashboard are immediately reflected in API responses, and vice versa. Your team can freely use both without conflicts.

1

Acknowledge the issue

  1. Open the issue detail panel
  2. Click the Acknowledge button
  3. The issue status changes to Acknowledged and records who acknowledged it
2

Fix the underlying problem

Apply the changes suggested by the analysis (e.g., increase timeout, add fallback model, update configuration).

3

Resolve the issue

  1. Return to the issue detail panel
  2. Click the Resolve button
  3. The issue status changes to Resolved and records who resolved it and when

Track acknowledgment and resolution times to measure your team’s incident response performance over time.

Troubleshooting

IssueSolution
No issues are being createdVerify monitor status is “active”. Check that the trigger interval and threshold are configured correctly. Ensure your assistants have recent call data.
Alerts not receivedConfirm notifiers are enabled on the trigger. Verify credential IDs reference valid notifiers. Check the notifier configuration (email address, webhook URL, Slack webhook).
Analysis returns an errorEnsure the issue has associated calls. Analysis requires call data to identify patterns.
Monitor not evaluatingCheck that the insightId references a valid Insight. Verify the trigger interval is at least 1 minute.
Wrong calls detectedReview the Insight query that the trigger references. Ensure the monitor targets the correct assistants.
Duplicate issuesTriggers create new issues per evaluation window. This is expected behavior when a problem persists across multiple evaluation periods.

If an alert shows a "failure" status in the issue’s alerts array, the notification delivery failed. Check your notifier configuration and ensure the destination (email, Slack webhook, URL) is reachable.

Next steps

Advanced monitoring

Configure schedule-based triggers, multi-threshold monitors, and compliance monitoring

Structured outputs

Use structured outputs with effectiveness and compliance monitors

Boards

Visualize monitoring data and call metrics on dashboards

Evals

Test your assistants before deployment with automated evaluations

Get help

Need assistance? We’re here to help: