Fine-tuned OpenAI models
Use Another LLM or Your Own Server
Vapi supports using any OpenAI-compatible endpoint as the LLM. This includes services like OpenRouter, AnyScale, Together AI, or your own server.
When to Use Custom LLMs
- For an open-source LLM, like Mixtral
- To update the context during the conversation
- To customize the messages before theyβre sent to an LLM
Using an LLM provider
Youβll first want to POST your API key via the /credential
endpoint:
Then, you can create an assistant with the model provider:
Using Fine-Tuned OpenAI Models
To set up your OpenAI Fine-Tuned model, you need to follow these steps:
- Set the custom llm URL to
https://api.openai.com/v1
. - Assign the custom llm key to the OpenAI key.
- Update the model to their model.
- Execute a PATCH request to the
/assistant
endpoint and ensure thatmodel.metadataSendMode
is set to off.
Using your server
To set up your server to act as the LLM, youβll need to create an endpoint that is compatible with the OpenAI Client. For best results, your endpoint should also support streaming completions.
If your server is making calls to an OpenAI compatble API, you can pipe the requests directly back in your response to Vapi.
If youβd like your OpenAI-compatible endpoint to be authenticated, you can POST your serverβs API key and URL via the /credential
endpoint:
If your server isnβt authenticated, you can skip this step.
Then, you can create an assistant with the custom-llm
model provider: