How to correctly stop and remove an audioTapProcessor in iOS11

问题

I'm using an audioTapProcessor similar to the audioTap example from apple.

Basically, the audioTapProcessor is created and added to the audioMix:

MTAudioProcessingTapRef audioProcessingTap;
if (noErr == MTAudioProcessingTapCreate(kCFAllocatorDefault, &callbacks,     kMTAudioProcessingTapCreationFlag_PreEffects, &audioProcessingTap))
{
    audioMixInputParameters.audioTapProcessor = audioProcessingTap;

    CFRelease(audioProcessingTap);

    audioMix.inputParameters = @[audioMixInputParameters];

    _audioMix = audioMix;
}

When the audioMix is created, it is added to the currentItem of the avPlayer:

self.avPlayer.currentItem.audioMix = audioMix;

When i'm done with the audioTapProcessor, I clean out references to the audioTapProcessor and the audioMix:

if (self.avPlayer.currentItem.audioMix){

    for (AVMutableAudioMixInputParameters *inputParameter in player.currentItem.audioMix.inputParameters) {
        inputParameter.audioTapProcessor = nil;

    }

    player.currentItem.audioMix = nil;
}

This triggers the tap_UnprepareCallback and the tap_FinalizeCallback functions, and the audioTapProcessor objects get's deallocated.

This worked without any issues until iOS11 came out a couple of days ago.

On iOS11 devices, I sometimes receive an EXC_BAD_ACCESS error on the ClientProcessingTapManager Thread. The error occurs in one of the classes of AVFoundation, so I can't debug. The error occurs somewhere in ClientProcessingTap::PerformTap.

I have added logging to the audioTap callbacks, and it looks like only the tap_ProcessCallBack is executed on the ClientProcessingTapManager thread.

I think that sometimes the tap_ProcessCallback method of an audioTapProcessor is triggered on this thread, while the actual audioTapProcessor is already deallocated on another Thread.

My question is: how can I correctly stop an audioTapProcessor and remove it from memory?

This is some example output logging:

2017-09-22 12:49:26:171 xx[1261:780894] cleaning up audioMix of AvPlayer
2017-09-22 12:49:26:213 xx[1261:781173] into tap_ProcessCallback. thread: ClientProcessingTapManager
2017-09-22 12:49:26:213 xx[1261:781173] end of tap_ProcessCallback
2017-09-22 12:49:26:217 xx[1261:781127] into tap_UnprepareCallback. 
2017-09-22 12:49:26:218 xx[1261:781127] end of tap_UnprepareCallback
2017-09-22 12:49:26:218 xx[1261:781127] into tap_FinalizeCallback. 
2017-09-22 12:49:26:218 Tunify[1261:781127] end of tap_FinalizeCallback
2017-09-22 12:49:26:218 Tunify[1261:781127] into MYAudioTapProcessor dealloc.
(lldb)

Good to know:

I'm usng a pool of avPlayers, so i'm reusing these objects. When I don't clean out the audioMix of my avPlayers I don't run into this error. Without the cleaning up of the audioMix objects, the audioTapProcessors are only deallocated when I reuse an avPlayer and assign a new audioMix to the avPlayer.

I found a related question, but i'm already triggering the cleanup functions. The solution there gives the same error on iOS11 devices

回答1:

This is a bug in AVFoundation under iOS 11. It has been fixed in iOS 12. A radar for the issue rdar://34977000

A possible workaround is to call prepareReleaseAudioMix: just before releasing the AVPlayer / AVPlayerItem

+ (void)prepareReleaseAudioMix:(AVAudioMix*)audioMix {
    if ( @available(iOS 11.0, *) ) {
        // Starting with iOS 11, it is not required to manually nil audioTapProcessor,
        // but we need to retain the audioMix for a bit to make sure the processing callback
        // will not be called after we release (this is due to a bug in iOS 11 which calls the release
        // callback but still calls the processing callback afterwards - it also releases internal data
        // on release, so simply checking for release in the processing block is not enough)
        // rdar://34977000
        dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(1 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
            [self _prepareReleaseAudioMixNow:audioMix];
        });
    } else {
        // Prior to iOS 11 we need to manually nil the audioTapProcessor
        [self _prepareReleaseAudioMixNow:audioMix];
    }
}

+ (void)_prepareReleaseAudioMixNow:(AVAudioMix*)audioMix {
    for ( AVMutableAudioMixInputParameters *parameters in audioMix.inputParameters ) {
        if ( ![parameters isKindOfClass:[AVMutableAudioMixInputParameters class]] )
            continue;
        parameters.audioTapProcessor = nil;
    }
}

来源：https://stackoverflow.com/questions/46362859/how-to-correctly-stop-and-remove-an-audiotapprocessor-in-ios11

标签

ios

objective-c

avfoundation