Background speech denoising

Overview

Background speech denoising helps create clearer conversations by filtering out unwanted sounds while users speak. Vapi offers two complementary denoising technologies that can be used independently or together for optimal results.

In this guide, you’ll learn to:

• Enable Smart Denoising using Krisp technology (recommended for most users)
• Configure experimental Fourier denoising with customizable parameters
• Combine both methods for enhanced noise reduction
• Fine-tune settings for different environments

Denoising methods

Smart Denoising (Krisp)

Smart Denoising uses Krisp’s AI-powered technology to remove background noise in real-time. This method is highly effective for common noise sources like:

• Keyboard typing
• Background conversations
• Traffic and street noise
• Air conditioning and fans
• Pet sounds

Fourier Denoising (Experimental)

Fourier denoising uses frequency-domain filtering to remove consistent background noise. This experimental method offers fine-grained control through multiple parameters and includes automatic media detection for TV/music/radio backgrounds.

Configuration

Background speech denoising is configured through the backgroundSpeechDenoisingPlan property on your assistant:

1
import { VapiClient } from "@vapi-ai/server-sdk";
2
3
const vapi = new VapiClient({ 
4
  token: process.env.VAPI_API_KEY 
5
});
6
7
const assistant = await vapi.assistants.create({
8
  name: "Customer Support",
9
  backgroundSpeechDenoisingPlan: {
10
    // Enable Smart Denoising
11
    smartDenoisingPlan: {
12
      enabled: true
13
    },
14
    // Enable Fourier Denoising (optional)
15
    fourierDenoisingPlan: {
16
      enabled: true,
17
      mediaDetectionEnabled: true,
18
      staticThreshold: -35,
19
      baselineOffsetDb: -15,
20
      windowSizeMs: 3000,
21
      baselinePercentile: 85
22
    }
23
  }
24
});

Smart Denoising configuration

Smart Denoising has a simple on/off configuration:

Example: Smart Denoising only

1
const assistant = await vapi.assistants.create({
2
  name: "Support Agent",
3
  backgroundSpeechDenoisingPlan: {
4
    smartDenoisingPlan: {
5
      enabled: true
6
    }
7
  }
8
});

Fourier Denoising configuration

Fourier denoising offers multiple parameters for fine-tuning:

Example: Adding Fourier Denoising to Smart Denoising

1
const assistant = await vapi.assistants.create({
2
  name: "Call Center Agent",
3
  backgroundSpeechDenoisingPlan: {
4
    // Always enable Smart Denoising first
5
    smartDenoisingPlan: {
6
      enabled: true
7
    },
8
    // Add Fourier Denoising for additional filtering
9
    fourierDenoisingPlan: {
10
      enabled: true,
11
      mediaDetectionEnabled: true,
12
      // More aggressive filtering for noisy environments
13
      baselineOffsetDb: -10,
14
      // Faster adaptation for dynamic environments
15
      windowSizeMs: 2000,
16
      // Focus on louder, clearer speech
17
      baselinePercentile: 90
18
    }
19
  }
20
});

Combined denoising

For maximum noise reduction, combine both methods. Processing order:

1. Smart Denoising (Krisp) processes first
2. Fourier Denoising processes the Krisp output

Environment-specific configurations

Quiet office environment

Minimal speech denoising for clear environments:

1
const assistant = await vapi.assistants.create({
2
  name: "Office Assistant",
3
  backgroundSpeechDenoisingPlan: {
4
    smartDenoisingPlan: {
5
      enabled: true
6
    }
7
    // No Fourier denoising needed
8
  }
9
});

Noisy call center

Aggressive filtering for high-noise environments:

1
const assistant = await vapi.assistants.create({
2
  name: "Call Center Agent",
3
  backgroundSpeechDenoisingPlan: {
4
    smartDenoisingPlan: {
5
      enabled: true
6
    },
7
    fourierDenoisingPlan: {
8
      enabled: true,
9
      mediaDetectionEnabled: true,
10
      baselineOffsetDb: -10, // Aggressive filtering
11
      windowSizeMs: 2000,    // Fast adaptation
12
      baselinePercentile: 90 // Focus on clear speech
13
    }
14
  }
15
});

Home environment with TV/music

Optimized for media background noise:

1
const assistant = await vapi.assistants.create({
2
  name: "Home Assistant",
3
  backgroundSpeechDenoisingPlan: {
4
    smartDenoisingPlan: {
5
      enabled: true
6
    },
7
    fourierDenoisingPlan: {
8
      enabled: true,
9
      mediaDetectionEnabled: true, // Essential for TV/music
10
      baselineOffsetDb: -15,
11
      windowSizeMs: 4000,
12
      baselinePercentile: 80
13
    }
14
  }
15
});

Best practices

When to use each method

Smart Denoising only:

• General-purpose noise reduction
• Unpredictable noise patterns
• When simplicity is preferred

Smart Denoising + Fourier Denoising:

• Maximum noise reduction required
• Consistent background noise that Smart Denoising alone cannot fully handle
• Complex acoustic environments with media (TV/music/radio)
• Premium user experiences requiring fine-tuned control
• Willing to invest time in testing and tuning
• Not using headphones (Fourier may cause issues with headphone audio)

Performance considerations

Audio quality: Aggressive filtering may affect voice quality. Test different settings to find the right balance between noise reduction and natural speech preservation.

Testing recommendations

1. Test in your target environment
2. Start with default settings
3. Adjust parameters incrementally
4. Monitor user feedback
5. A/B test different configurations

Troubleshooting fourier denoising