> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.vapi.ai/llms.txt.
> For full documentation content, see https://docs.vapi.ai/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.vapi.ai/_mcp/server.

# Fine-tuned OpenAI models

Vapi supports using any OpenAI-compatible endpoint as the LLM. This includes services like [OpenRouter](https://openrouter.ai/), [AnyScale](https://www.anyscale.com/), [Together AI](https://www.together.ai/), or your own server.

* For an open-source LLM, like Mixtral
* To update the context during the conversation
* To customize the messages before they're sent to an LLM

## Using an LLM provider

You'll first want to POST your API key via the `/credential` endpoint:

```json
{
  "provider": "openrouter",
  "apiKey": "<YOUR OPENROUTER KEY>"
}
```

Then, you can create an assistant with the model provider:

```json
{
  "name": "My Assistant",
  "model": {
    "provider": "openrouter",
    "model": "cognitivecomputations/dolphin-mixtral-8x7b",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ],
    "temperature": 0.7
  }
}
```

## Using Fine-Tuned OpenAI Models

To set up your OpenAI Fine-Tuned model, you need to follow these steps:

1. Set the custom llm URL to `https://api.openai.com/v1`.
2. Assign the custom llm key to the OpenAI key.
3. Update the model to their model.
4. Execute a PATCH request to the `/assistant` endpoint and ensure that `model.metadataSendMode` is set to off.

## Using your server

To set up your server to act as the LLM, you'll need to create an endpoint that is compatible with the [OpenAI Client](https://platform.openai.com/docs/api-reference/making-requests). For best results, your endpoint should also support streaming completions.

If your server is making calls to an OpenAI compatble API, you can pipe the requests directly back in your response to Vapi.

If you'd like your OpenAI-compatible endpoint to be authenticated, you can POST your server's API key and URL via the `/credential` endpoint:

```json
{
  "provider": "custom-llm",
  "apiKey": "<YOUR SERVER API KEY>"
}
```

If your server isn't authenticated, you can skip this step.

Then, you can create an assistant with the `custom-llm` model provider:

```json
{
  "name": "My Assistant",
  "model": {
    "provider": "custom-llm",
    "url": "<YOUR OPENAI COMPATIBLE ENDPOINT BASE URL>",
    "model": "my-cool-model",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ],
    "temperature": 0.7
  }
}
```