Documentation agent

Try our live implementation using the voice widget in the bottom-right corner of this page.

Overview

Build a voice-powered documentation assistant step by step. Use API requests (shown below) or the Vapi Dashboard.

You’ll learn to:

Index your docs with LlamaCloud
Create a RAG tool for document retrieval
Create an assistant with Claude 3.5 Sonnet and attach the tool
Use the Web SDK to create a widget
Analyze user sessions and improve the quality of your agent overtime

Prerequisites

Vapi account with API access
Documentation content - use your llms-full.txt file (example), available out-of-box on Fern or Mintlify
LlamaCloud account for indexing

Get started

Index your documentation

Upload and index your documentation in LlamaCloud using text-embedding-3-small.

Create a new project in LlamaCloud
Upload your documentation files (you can use a single consolidated file like llms-full.txt)
Configure embedding model to text-embedding-3-small
Set chunking to 512 tokens with 50 token overlap
Note your index ID and API credentials

Consolidate your documentation into a single text file for better RAG performance. You can see our example at docs.vapi.ai/llms-full.txt.

Create the RAG tool

Create a tool that connects your assistant to your LlamaCloud index using the Tools API.

You can also create this tool in the Vapi Dashboard.

$ curl -X POST https://api.vapi.ai/tool \
>      -H "Authorization: Bearer YOUR_VAPI_API_KEY" \
>      -H "Content-Type: application/json" \
>      -d '{
>        "type": "apiRequest",
>        "name": "docsquery",
>        "function": {
>          "name": "docsquery",
>          "parameters": {
>            "type": "object",
>            "properties": {
>              "query": {
>                "type": "string",
>                "description": "The search query to find relevant documentation"
>              }
>            },
>            "required": ["query"]
>          }
>        },
>        "url": "https://api.cloud.llamaindex.ai/api/v1/pipelines/YOUR_PIPELINE_ID/retrieve",
>        "method": "POST",
>        "headers": {
>          "type": "object",
>          "properties": {
>            "Content-Type": {
>              "type": "string",
>              "value": "application/json"
>            },
>            "Authorization": {
>              "type": "string",
>              "value": "Bearer YOUR_LLAMACLOUD_API_KEY"
>            }
>          }
>        },
>        "body": {
>          "type": "object",
>          "properties": {
>            "query": {
>              "type": "string",
>              "value": "{{query}}"
>            }
>          }
>        }
>      }'

Replace YOUR_PIPELINE_ID with your LlamaCloud pipeline ID and YOUR_LLAMACLOUD_API_KEY with your API key. Save the tool ID from the response for the next step.

Create an assistant with the tool

Create an assistant using the Assistant API with the RAG tool attached.

You can also create this assistant in the Vapi Dashboard.

$ curl -X POST https://api.vapi.ai/assistant \
>      -H "Authorization: Bearer YOUR_VAPI_API_KEY" \
>      -H "Content-Type: application/json" \
>      -d '{
>        "name": "Docs agent",
>        "model": {
>          "provider": "anthropic",
>          "model": "claude-3-5-sonnet-20241022",
>          "maxTokens": 400,
>          "messages": [
>            {
>              "role": "system",
>              "content": "You are a helpful documentation assistant. Use the docsquery tool to find relevant information. Always be helpful, friendly, and concise. Provide accurate information based on the documentation. When you don'\''t know something, say so clearly. Keep responses conversational for voice interaction."
>            }
>          ],
>          "toolIds": ["YOUR_TOOL_ID_FROM_STEP_2"]
>        },
>        "voice": {
>          "provider": "vapi",
>          "voiceId": "Harry"
>        },
>        "transcriber": {
>          "provider": "deepgram",
>          "model": "nova-3",
>          "language": "en"
>        },
>        "firstMessage": "Hey I'\''m Harry, a support agent. How can I help you today? You can ask me questions about Vapi, how to get started or our documentation.",
>        "endCallMessage": "Goodbye.",
>        "backgroundSound": "off",
>        "analysisPlan": {
>          "summaryPlan": {
>            "enabled": true,
>            "prompt": "Summarize this documentation support call, focusing on the user'\''s questions and how well they were answered."
>          },
>          "successEvaluationPlan": {
>            "enabled": true,
>            "prompt": "Evaluate if this documentation support call was successful. Did the user get helpful answers to their questions?",
>            "rubric": "NumericScale"
>          }
>        }
>      }'

Replace YOUR_TOOL_ID_FROM_STEP_2 with the tool ID from step 2. Save the assistant ID from the response for the next step.

See our complete documentation agent prompt that includes detailed personality, style guidelines, and interaction patterns.

Create a web component

Use the Vapi Web SDK to create a voice widget.

$ npm install @vapi-ai/web

Replace YOUR_PUBLIC_API_KEY and YOUR_ASSISTANT_ID with your actual values:

1 import { useState, useEffect } from 'react';
2 import Vapi from '@vapi-ai/web';
3 
4 export default function VoiceWidget() {
5   const [vapi, setVapi] = useState(null);
6   const [isConnected, setIsConnected] = useState(false);
7   const [transcript, setTranscript] = useState([]);
8 
9   useEffect(() => {
10     const vapiInstance = new Vapi('YOUR_PUBLIC_API_KEY');
11     setVapi(vapiInstance);
12 
13     vapiInstance.on('call-start', () => setIsConnected(true));
14     vapiInstance.on('call-end', () => setIsConnected(false));
15     vapiInstance.on('message', (msg) => {
16       if (msg.type === 'transcript') {
17         setTranscript(prev => [...prev, { role: msg.role, text: msg.transcript }]);
18       }
19     });
20 
21     return () => vapiInstance?.stop();
22   }, []);
23 
24   const startCall = () => vapi?.start('YOUR_ASSISTANT_ID');
25   const endCall = () => vapi?.stop();
26 
27   return (
28     <div style={{ position: 'fixed', bottom: 24, right: 24, background: '#000', color: '#fff', borderRadius: 12, padding: 20, width: 300 }}>
29       {!isConnected ? (
30         <button onClick={startCall} style={{ background: '#12A594', color: '#fff', border: 'none', borderRadius: 8, padding: '12px 24px' }}>
31           Start Voice Chat
32         </button>
33       ) : (
34         <div>
35           <div style={{ maxHeight: 200, overflowY: 'auto', marginBottom: 16 }}>
36             {transcript.map((msg, i) => (
37               <div key={i} style={{ marginBottom: 8, textAlign: msg.role === 'user' ? 'right' : 'left' }}>
38                 <span style={{ background: msg.role === 'user' ? '#12A594' : '#333', padding: '8px 12px', borderRadius: 12, display: 'inline-block' }}>
39                   {msg.text}
40                 </span>
41               </div>
42             ))}
43           </div>
44           <button onClick={endCall} style={{ background: '#e5e7eb', color: '#000', border: 'none', borderRadius: 8, padding: '8px 16px' }}>
45             End Call
46           </button>
47         </div>
48       )}
49     </div>
50   );
51 }

For a complete implementation with waveform visualization, real-time transcripts, and responsive design, check out our voice widget component on GitHub.

Improve your prompts with call analysis

Vapi automatically analyzes every call. The assistant above includes an analysisPlan with summary and success evaluation configured.

Configure additional analysis options in your assistant:

Summary plan: Custom prompts for call summaries
Structured data plan: Extract specific information using JSON schemas
Success evaluation plan: Score calls with custom rubrics
Structured data multi plan: Multiple extraction schemas

Retrieve analysis results using the Get Call API:

$ curl https://api.vapi.ai/call/{CALL_ID} \
>      -H "Authorization: Bearer YOUR_VAPI_API_KEY"

The response includes call.analysis with your configured analysis results. Learn more about call analysis configuration.

Iterative improvements:

Review analysis summaries to identify common user questions
Use structured data to track conversation patterns
Monitor success evaluations to optimize assistant performance
Update your system prompt based on recurring issues
Refine RAG retrieval based on query success patterns

Prompting guide

Learn advanced prompting techniques to optimize your documentation agent’s responses and behavior.