How would you connect an iPod library asset to an Audio Queue Service and process with an Audio Unit?

Question

I need to process audio that comes from the iPod library. The only way to read an asset for the iPod library is AVAssetReader. To process audio with an Audio Unit it needs to be in stereo format so I have values for the left and right channels. But when I use AVAssetReader to read an asset from the iPod library it does not allow me to get it out in stereo format. It comes out in interleaved format which I do not know how to break into left and right audio channels.

To get to where I need to go I would need to do one of the following:

1. Get AVAssetReader to give me an AudioBufferList in stereo format
2. Convert the interleaved data to non-interleaved to get the stereo output I need
3. Send it through Audio Queue Services to get what I need with automatic buffering

I seem to be limited by what the existing public API can do and what AVAssetReader supports when reading an iPod library asset. What would you do? How can I get what I need to process with an Audio Unit?

Another limitation that I have is that I cannot read an entire song at a time as it would fill up memory and crash the app. This is the reason I want to use the Audio Queue Services. If I can treat the asset from the iPod library as a stream in stereo format then all my requirements would be addressed.

Can this even be done? Are there any docs, blogs or articles anywhere that would explain how this could be done?

Answer 1

Sounds like you have a couple questions stacked in there.

When you setups an AVAssetReader you can pass in a dictionary of settings. Here is how I create my AVAssetReaders...

    AVAssetReader* CreateAssetReaderFromSong(AVURLAsset* songURL) {

    if([songURL.tracks count] <= 0)
        return NULL;


    AVAssetTrack* songTrack = [songURL.tracks objectAtIndex:0];

    NSDictionary* outputSettingsDict = [[NSDictionary alloc] initWithObjectsAndKeys:

                                        [NSNumber numberWithInt:kAudioFormatLinearPCM],AVFormatIDKey,
                                        //     [NSNumber numberWithInt:AUDIO_SAMPLE_RATE],AVSampleRateKey,  /*Not Supported*/
                                        //     [NSNumber numberWithInt: 2],AVNumberOfChannelsKey,   /*Not Supported*/

                                        [NSNumber numberWithInt:16],AVLinearPCMBitDepthKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsBigEndianKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsFloatKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsNonInterleaved,

                                        nil];

    NSError* error = nil;
    AVAssetReader* reader = [[AVAssetReader alloc] initWithAsset:songURL error:&error];

    {
        AVAssetReaderTrackOutput* output = [[AVAssetReaderTrackOutput alloc] initWithTrack:songTrack outputSettings:outputSettingsDict];
        [reader addOutput:output];
        [output release];
    }

    return reader;
}

So as far as splitting the Left and Right channel, you can loop over the data based on your 'AVLinearPCMBitDepthKey'.

So something like this for 16 bit...

for (j=0; j<tBufCopy; j++, pAD+=2) {            // Fill the buffers...
    mProcessingBuffer.Left[(tBlockUsed+j)] = ((sint32)pAD[0]);
    mProcessingBuffer.Right[(tBlockUsed+j)] = ((sint32)pAD[1]);
}

Now I assume you need this for your processing. But having the data in interleaved format is really quite nice. You can generally take the straight interleaved format and pass it right back to the AudioQueue or Remote I/O callback and it will play correctly.

In order to get the audio playing using the AudioQueue framework the data should follow this flow:

AVAssetReader -> NSData Buffer -> AudioQueueBuffer

Then in the AudioQueue callback where it asks for more data simply pass the AudioQueueBuffer. Something like...

- (void) audioQueueCallback:(AudioQueueRef)aq  buffer:(AudioQueueBufferRef)buffer {

    memcpy(buffer->mAudioData, srcData, mBufferByteSize);
    //Setup buffer->mAudioDataSize

    //...

    AudioQueueEnqueueBuffer(mQueue, buffer, 0 /*CBR*/, 0 /*non compressed*/);
}