WebSocket Transport

Vapi’s WebSocket transport enables real-time, bidirectional audio communication directly between your application and Vapi’s AI assistants. Unlike traditional phone or web calls, this transport method lets you stream raw audio data instantly with minimal latency.

Key Benefits

• Low Latency: Direct streaming ensures minimal delays.
• Bidirectional Streaming: Real-time audio flow in both directions.
• Easy Integration: Compatible with any environment supporting WebSockets.
• Flexible Audio Formats: Customize audio parameters such as sample rate.
• Automatic Sample Rate Conversion: Seamlessly handles various audio rates.

Creating a WebSocket Call

To initiate a call using WebSocket transport:

PCM Format (16-bit, default)

$
curl 'https://api.vapi.ai/call' \
>
  -H 'authorization: Bearer YOUR_API_KEY' \
>
  -H 'content-type: application/json' \
>
  --data-raw '{
>
    "assistantId": "YOUR_ASSISTANT_ID",
>
    "transport": {
>
      "provider": "vapi.websocket",
>
      "audioFormat": {
>
        "format": "pcm_s16le",
>
        "container": "raw",
>
        "sampleRate": 16000
>
      }
>
    }
>
  }'

Mu-Law Format

$
curl 'https://api.vapi.ai/call' \
>
  -H 'authorization: Bearer YOUR_API_KEY' \
>
  -H 'content-type: application/json' \
>
  --data-raw '{
>
    "assistantId": "YOUR_ASSISTANT_ID",
>
    "transport": {
>
      "provider": "vapi.websocket",
>
      "audioFormat": {
>
        "format": "mulaw",
>
        "container": "raw",
>
        "sampleRate": 8000
>
      }
>
    }
>
  }'

Sample API Response

1
{
2
  "id": "7420f27a-30fd-4f49-a995-5549ae7cc00d",
3
  "assistantId": "5b0a4a08-133c-4146-9315-0984f8c6be80",
4
  "type": "vapi.websocketCall",
5
  "createdAt": "2024-09-10T11:14:12.339Z",
6
  "updatedAt": "2024-09-10T11:14:12.339Z",
7
  "orgId": "eb166faa-7145-46ef-8044-589b47ae3b56",
8
  "cost": 0,
9
  "status": "queued",
10
  "transport": {
11
    "provider": "vapi.websocket",
12
    "websocketCallUrl": "wss://api.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/transport"
13
  }
14
}

Audio Format Configuration

When creating a WebSocket call, the audio format can be customized:

Supported Audio Formats

Vapi supports the following audio formats:

• pcm_s16le: 16-bit PCM, signed little-endian (default)
• mulaw: Mu-Law encoded audio (ITU-T G.711 standard)

Both formats use the raw container format for direct audio streaming.

Format Selection Guidelines

• PCM (pcm_s16le): Higher quality audio, larger bandwidth usage. Ideal for high-quality applications.
• Mu-Law (mulaw): Lower bandwidth, telephony-standard encoding. Ideal for telephony integrations and bandwidth-constrained environments.

Connecting to the WebSocket

Use the WebSocket URL from the response to establish a connection:

1
const socket = new WebSocket("wss://api.vapi.ai/7420f27a-30fd-4f49-a995-5549ae7cc00d/transport");
2
3
socket.onopen = () => console.log("WebSocket connection opened.");
4
socket.onclose = () => console.log("WebSocket connection closed.");
5
socket.onerror = (error) => console.error("WebSocket error:", error);

Sending and Receiving Data

The WebSocket supports two types of messages:

• Binary audio data (format depends on your configuration: PCM or Mu-Law)
• Text-based JSON control messages

Audio Data Format

The binary audio data format depends on your audioFormat configuration:

• PCM (pcm_s16le): 16-bit signed little-endian samples
• Mu-Law (mulaw): 8-bit Mu-Law encoded samples (ITU-T G.711)

Sending Audio Data

1
function sendAudioChunk(audioBuffer) {
2
  if (socket.readyState === WebSocket.OPEN) {
3
    socket.send(audioBuffer);
4
  }
5
}
6
7
navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
8
  const audioContext = new AudioContext();
9
  const source = audioContext.createMediaStreamSource(stream);
10
  const processor = audioContext.createScriptProcessor(1024, 1, 1);
11
12
  processor.onaudioprocess = (event) => {
13
    const pcmData = event.inputBuffer.getChannelData(0);
14
    const int16Data = new Int16Array(pcmData.length);
15
16
    for (let i = 0; i < pcmData.length; i++) {
17
      int16Data[i] = Math.max(-32768, Math.min(32767, pcmData[i] * 32768));
18
    }
19
20
    sendAudioChunk(int16Data.buffer);
21
  };
22
23
  source.connect(processor);
24
  processor.connect(audioContext.destination);
25
});

Receiving Data

1
socket.onmessage = (event) => {
2
  if (event.data instanceof Blob) {
3
    event.data.arrayBuffer().then(buffer => {
4
      const audioData = new Int16Array(buffer);
5
      playAudio(audioData);
6
    });
7
  } else {
8
    try {
9
      const message = JSON.parse(event.data);
10
      handleControlMessage(message);
11
    } catch (error) {
12
      console.error("Failed to parse message:", error);
13
    }
14
  }
15
};

Sending Control Messages

1
function sendControlMessage(messageObj) {
2
  if (socket.readyState === WebSocket.OPEN) {
3
    socket.send(JSON.stringify(messageObj));
4
  }
5
}
6
7
// Example: hangup call
8
function hangupCall() {
9
  sendControlMessage({ type: "hangup" });
10
}

Ending the Call

The recommended way to end a call is using Live Call Control which provides more control and proper cleanup.

Alternatively, you can end the WebSocket call directly:

1
sendControlMessage({ type: "end-call" });
2
socket.close();

Comparison: WebSocket Transport vs. Call Listen Feature

Vapi provides two WebSocket options:

Refer to Live Call Control for more on the Call Listen feature.