How do I make AVCaptureSession and AVPlayer respect AVAudioSessionCategoryAmbient?

Question

I'm making an app that records (AVCaptureSession) and plays (AVPlayerLayer) video. I'd like to be able to do this without pausing background audio from other apps and I'd like the playback to respect the mute switch.

In the AppDelegate I have set AVAudioSessionCategoryAmbient, according to the docs this should:

The category for an app in which sound playback is nonprimary—that is, your app can be used successfully with the sound turned off.

This category is also appropriate for “play along” style apps, such as a virtual piano that a user plays while the Music app is playing. When you use this category, audio from other apps mixes with your audio. Your audio is silenced by screen locking and by the Silent switch (called the Ring/Silent switch on iPhone).

This perfectly describes the behavior I'm looking for. But it doesn't work.

I know it's set because if I try print(AVAudioSession.sharedInstance().category) in any view controller it returns AVAudioSessionCategoryAmbient.

Any ideas? I'm using Swift but even a vague direction to look in would be appreciated.

Answer 1

How to Mix Background Audio With an AVCapture Session:

If you have a microphone input, an AVCapture session—by default—will set your apps AVAudioSession to AVAudioSessionCategoryPlayAndRecord. You've got to tell it not to:

AVCaptureSession.automaticallyConfiguresApplicationAudioSession = false

Doing this, however, just froze the app. Because unfortunately, AVAudioSessionCategoryAmbient just doesn't work with AVCaptureSession.

The solution is to set your apps AVAudioSession to AVAudioSessionCategoryPlayAndRecord with options:

do {
    try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord, withOptions: [.MixWithOthers, .AllowBluetooth, .DefaultToSpeaker])
    try AVAudioSession.sharedInstance().setActive(true)

} catch let error as NSError {
    print(error)
}

.MixWithOthers was kind of the most important one. That let the audio from other apps play. But it switched it to coming out of the earpiece which was super odd (I thought it was getting ducked at first). So .DefaultToSpeaker moves it to the bottom speaker and .AllowBluetooth lets you keep bluetooth audio coming out of headphones but also enables a bluetooth mic. Not sure if this can be refined anymore but they seemed like all the relevant options.

How to Respect the Mute Switch in Playback:

During recording, you set your AVAudioSession to AVAudioSessionCategoryPlayAndRecord, but that doesn't respect the mute switch.

Because you can't set AVAudioSessionCategoryAmbient when there's a microphone input. The trick is to remove the mic from the AVCaptureSession, then set the AVAudioSession to AVAudioSessionCategoryAmbient:

do {
    captureSession.removeInput(micInput)
    try AVAudioSession.sharedInstance().setActive(false)
    try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryAmbient)
    try AVAudioSession.sharedInstance().setActive(true)
} catch let error as NSError { print(error) }

Once you have finished playback and need to go back to recording, you need to set AVAudioSessionCategoryPlayAndRecord again (with options again so the background audio continues):

do {
    try AVAudioSession.sharedInstance().setActive(false)
    try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayAndRecord, withOptions: [.MixWithOthers, .AllowBluetooth, .DefaultToSpeaker])
    try AVAudioSession.sharedInstance().setActive(true)
} catch let error as NSError { print(error) }

captureSession.automaticallyConfiguresApplicationAudioSession = false
captureSession.addInput(micInput!)

The first line in the do block was the thing that had me caught up for a long time. I didn't need to set the audio session to inactive to switch to AVAudioSessionCategoryAmbient but it was pausing background audio when coming back to AVAudioSessionCategoryPlayAndRecord.