Call Handling with Vapi and Twilio

This document explains how to handle a scenario where a user is on hold while the system attempts to connect them to a specialist. If the specialist does not pick up within X seconds or if the call hits voicemail, we take an alternate action (like playing an announcement or scheduling an appointment). This solution integrates Vapi.ai for AI-driven conversations and Twilio for call bridging.

Problem

Vapi.ai does not provide a built-in way to keep the user on hold, dial a specialist, and handle cases where the specialist is unavailable. We want:

1. The user already talking to the AI (Vapi).
2. The AI offers to connect them to a specialist.
3. The user is placed on hold or in a conference room.
4. We dial the specialist to join.
5. If the specialist answers, everyone is merged.
6. If the specialist does not answer (within X seconds or goes to voicemail), we want to either announce “Specialist not available” or schedule an appointment.

Solution

1. An inbound call arrives from Vapi or from the user directly.
2. We store its details (e.g., Twilio CallSid).
3. We send TwiML (or instructions) to put the user in a Twilio conference (on hold).
4. We place a second call to the specialist, also directed to join the same conference.
5. If the specialist picks up, Twilio merges the calls.
6. If not, we handle the no-answer event by playing a message or returning control to the AI for scheduling.

Steps to Solve the Problem

1. Receive Inbound Call

   • Twilio posts data to your /inbound_call.
   • You store the call reference.
   • You might also invoke Vapi for initial AI instructions.

2. Prompt User via Vapi

   • The user decides whether they want the specialist.
   • If yes, you call an endpoint (e.g., /connect).

3. Create/Join Conference

   • In /connect, you update the inbound call to go into a conference route.
   • The user is effectively on hold.

4. Dial Specialist

   • You create a second call leg to the specialist’s phone.
   • A statusCallback can detect no-answer or voicemail.

5. Detect Unanswered

   • If Twilio sees a no-answer or failure, your callback logic plays an announcement or signals the AI to schedule an appointment.

6. Merge or Exit

   • If the specialist answers, they join the user.
   • If not, the user is taken off hold and the call ends or goes back to AI.

7. Use Ephemeral Call (Optional)

   • If you need an in-conference announcement, create a short-lived Twilio call that <Say> the message to everyone, then ends the conference.

Code Example

Below is a minimal Express.js server aligned for On-Hold Specialist Transfer with Vapi and Twilio.

1. Express Setup and Environment

1
const express = require("express");
2
const bodyParser = require("body-parser");
3
const axios = require("axios");
4
const twilio = require("twilio");
5
6
const app = express();
7
app.use(bodyParser.urlencoded({ extended: true }));
8
app.use(bodyParser.json());
9
10
// Load important env vars
11
const {
12
  TWILIO_ACCOUNT_SID,
13
  TWILIO_AUTH_TOKEN,
14
  FROM_NUMBER,
15
  TO_NUMBER,
16
  VAPI_BASE_URL,
17
  PHONE_NUMBER_ID,
18
  ASSISTANT_ID,
19
  PRIVATE_API_KEY,
20
} = process.env;
21
22
// Create a Twilio client
23
const client = twilio(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN);
24
25
// We'll store the inbound call SID here for simplicity
26
let globalCallSid = "";

2. /inbound_call - Handling the Inbound Call

1
app.post("/inbound_call", async (req, res) => {
2
  try {
3
    globalCallSid = req.body.CallSid;
4
    const caller = req.body.Caller;
5
6
    // Example: We call Vapi.ai to get initial TwiML
7
    const response = await axios.post(
8
      `${VAPI_BASE_URL || "https://api.vapi.ai"}/call`,
9
      {
10
        phoneNumberId: PHONE_NUMBER_ID,
11
        phoneCallProviderBypassEnabled: true,
12
        customer: { number: caller },
13
        assistantId: ASSISTANT_ID,
14
      },
15
      {
16
        headers: {
17
          Authorization: `Bearer ${PRIVATE_API_KEY}`,
18
          "Content-Type": "application/json",
19
        },
20
      }
21
    );
22
23
    const returnedTwiml = response.data.phoneCallProviderDetails.twiml;
24
    return res.type("text/xml").send(returnedTwiml);
25
  } catch (err) {
26
    return res.status(500).send("Internal Server Error");
27
  }
28
});

3. /connect - Putting User on Hold and Dialing Specialist

1
app.post("/connect", async (req, res) => {
2
  try {
3
    const protocol =
4
      req.headers["x-forwarded-proto"] === "https" ? "https" : "http";
5
    const baseUrl = `${protocol}://${req.get("host")}`;
6
    const conferenceUrl = `${baseUrl}/conference`;
7
8
    // 1) Update inbound call to fetch TwiML from /conference
9
    await client.calls(globalCallSid).update({
10
      url: conferenceUrl,
11
      method: "POST",
12
    });
13
14
    // 2) Dial the specialist
15
    const statusCallbackUrl = `${baseUrl}/participant-status`;
16
17
    await client.calls.create({
18
      to: TO_NUMBER,
19
      from: FROM_NUMBER,
20
      url: conferenceUrl,
21
      method: "POST",
22
      statusCallback: statusCallbackUrl,
23
      statusCallbackMethod: "POST",
24
    });
25
26
    return res.json({ status: "Specialist call initiated" });
27
  } catch (err) {
28
    return res.status(500).json({ error: "Failed to connect specialist" });
29
  }
30
});

4. /conference - Placing Callers Into a Conference

1
app.post("/conference", (req, res) => {
2
  const VoiceResponse = twilio.twiml.VoiceResponse;
3
  const twiml = new VoiceResponse();
4
5
  // Put the caller(s) into a conference
6
  const dial = twiml.dial();
7
  dial.conference(
8
    {
9
      startConferenceOnEnter: true,
10
      endConferenceOnExit: true,
11
    },
12
    "my_conference_room"
13
  );
14
15
  return res.type("text/xml").send(twiml.toString());
16
});

5. /participant-status - Handling No-Answer or Busy

1
app.post("/participant-status", async (req, res) => {
2
  const callStatus = req.body.CallStatus;
3
  if (["no-answer", "busy", "failed"].includes(callStatus)) {
4
    console.log("Specialist did not pick up:", callStatus);
5
    // Additional logic: schedule an appointment, ephemeral call, etc.
6
  }
7
  return res.sendStatus(200);
8
});

6. /announce (Optional) - Ephemeral Announcement

1
app.post("/announce", (req, res) => {
2
  const VoiceResponse = twilio.twiml.VoiceResponse;
3
  const twiml = new VoiceResponse();
4
  twiml.say("Specialist is not available. Ending call now.");
5
6
  // Join the conference, then end it.
7
  twiml.dial().conference(
8
    {
9
      startConferenceOnEnter: true,
10
      endConferenceOnExit: true,
11
    },
12
    "my_conference_room"
13
  );
14
15
  return res.type("text/xml").send(twiml.toString());
16
});

7. Starting the Server

1
app.listen(3000, () => {
2
  console.log("Server running on port 3000");
3
});

How to Test

1. Environment Variables
   Set TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, FROM_NUMBER, TO_NUMBER, VAPI_BASE_URL, PHONE_NUMBER_ID, ASSISTANT_ID, and PRIVATE_API_KEY.

2. Expose Your Server

   • Use a tool like ngrok to create a public URL to port 3000.
   • Configure your Twilio phone number to call /inbound_call when a call comes in.

3. Place a Real Call

   • Dial your Twilio number from a phone.
   • Twilio hits /inbound_call, and run Vapi logic.
   • Trigger /connect to conference the user and dial the specialist.
   • If the specialist answers, they join the same conference.
   • If they never answer, Twilio eventually calls /participant-status.

4. Use cURL for Testing

   • Simulate Inbound:

     $
     curl -X POST https://<public-url>/inbound_call \
     >
       -F "CallSid=CA12345" \
     >
       -F "Caller=+15551112222"
   • Connect:

     $
     curl -X POST https://<public-url>/connect \
     >
       -H "Content-Type: application/json" \
     >
       -d "{}"

Note on Replacing “Connect” with Vapi Tools

Vapi offers built-in functions or custom tool calls for placing a second call or transferring, you can replace the manual /connect call with that Vapi functionality. The flow remains the same: user is put in a Twilio conference, the specialist is dialed, and any no-answer events are handled.

Notes & Limitations

1. Voicemail
   If a phone’s voicemail picks up, Twilio sees it as answered. Consider advanced detection or a fallback.

2. Concurrent Calls
   Multiple calls at once require storing separate CallSids or similar references.

3. Conference Behavior
   startConferenceOnEnter: true merges participants immediately; endConferenceOnExit: true ends the conference when that participant leaves.

4. X Seconds
   Decide how you detect no-answer. Typically, Twilio sets a final callStatus if the remote side never picks up.

With these steps and code, you can integrate Vapi Assistant while using Twilio’s conferencing features to hold, dial out to a specialist, and handle an unanswered or unavailable specialist scenario.