How to play audio backwards?

Question

Some people suggested to read the audio data from end to start and create a copy written from start to end, and then simply play that reversed audio data.

Are there existing examples for iOS how this is done?

I found an example project called MixerHost, which at some point uses an AudioUnitSampleType holding the audio data that has been read from file, and assigning it to a buffer.

This is defined as:

typedef SInt32 AudioUnitSampleType;
#define kAudioUnitSampleFractionBits 24

And according to Apple:

The canonical audio sample type for audio units and other audio processing in iPhone OS is noninterleaved linear PCM with 8.24-bit fixed-point samples.

So in other words it holds noninterleaved linear PCM audio data.

But I can't figure out where this data is beeing read in, and where it is stored. Here's the code that loads the audio data and buffers it:

- (void) readAudioFilesIntoMemory {

    for (int audioFile = 0; audioFile < NUM_FILES; ++audioFile)  {

        NSLog (@"readAudioFilesIntoMemory - file %i", audioFile);

        // Instantiate an extended audio file object.
        ExtAudioFileRef audioFileObject = 0;

        // Open an audio file and associate it with the extended audio file object.
        OSStatus result = ExtAudioFileOpenURL (sourceURLArray[audioFile], &audioFileObject);

        if (noErr != result || NULL == audioFileObject) {[self printErrorMessage: @"ExtAudioFileOpenURL" withStatus: result]; return;}

        // Get the audio file's length in frames.
        UInt64 totalFramesInFile = 0;
        UInt32 frameLengthPropertySize = sizeof (totalFramesInFile);

        result =    ExtAudioFileGetProperty (
                        audioFileObject,
                        kExtAudioFileProperty_FileLengthFrames,
                        &frameLengthPropertySize,
                        &totalFramesInFile
                    );

        if (noErr != result) {[self printErrorMessage: @"ExtAudioFileGetProperty (audio file length in frames)" withStatus: result]; return;}

        // Assign the frame count to the soundStructArray instance variable
        soundStructArray[audioFile].frameCount = totalFramesInFile;

        // Get the audio file's number of channels.
        AudioStreamBasicDescription fileAudioFormat = {0};
        UInt32 formatPropertySize = sizeof (fileAudioFormat);

        result =    ExtAudioFileGetProperty (
                        audioFileObject,
                        kExtAudioFileProperty_FileDataFormat,
                        &formatPropertySize,
                        &fileAudioFormat
                    );

        if (noErr != result) {[self printErrorMessage: @"ExtAudioFileGetProperty (file audio format)" withStatus: result]; return;}

        UInt32 channelCount = fileAudioFormat.mChannelsPerFrame;

        // Allocate memory in the soundStructArray instance variable to hold the left channel, 
        //    or mono, audio data
        soundStructArray[audioFile].audioDataLeft =
            (AudioUnitSampleType *) calloc (totalFramesInFile, sizeof (AudioUnitSampleType));

        AudioStreamBasicDescription importFormat = {0};
        if (2 == channelCount) {

            soundStructArray[audioFile].isStereo = YES;
            // Sound is stereo, so allocate memory in the soundStructArray instance variable to  
            //    hold the right channel audio data
            soundStructArray[audioFile].audioDataRight =
                (AudioUnitSampleType *) calloc (totalFramesInFile, sizeof (AudioUnitSampleType));
            importFormat = stereoStreamFormat;

        } else if (1 == channelCount) {

            soundStructArray[audioFile].isStereo = NO;
            importFormat = monoStreamFormat;

        } else {

            NSLog (@"*** WARNING: File format not supported - wrong number of channels");
            ExtAudioFileDispose (audioFileObject);
            return;
        }

        // Assign the appropriate mixer input bus stream data format to the extended audio 
        //        file object. This is the format used for the audio data placed into the audio 
        //        buffer in the SoundStruct data structure, which is in turn used in the 
        //        inputRenderCallback callback function.

        result =    ExtAudioFileSetProperty (
                        audioFileObject,
                        kExtAudioFileProperty_ClientDataFormat,
                        sizeof (importFormat),
                        &importFormat
                    );

        if (noErr != result) {[self printErrorMessage: @"ExtAudioFileSetProperty (client data format)" withStatus: result]; return;}

        // Set up an AudioBufferList struct, which has two roles:
        //
        //        1. It gives the ExtAudioFileRead function the configuration it 
        //            needs to correctly provide the data to the buffer.
        //
        //        2. It points to the soundStructArray[audioFile].audioDataLeft buffer, so 
        //            that audio data obtained from disk using the ExtAudioFileRead function
        //            goes to that buffer

        // Allocate memory for the buffer list struct according to the number of 
        //    channels it represents.
        AudioBufferList *bufferList;

        bufferList = (AudioBufferList *) malloc (
            sizeof (AudioBufferList) + sizeof (AudioBuffer) * (channelCount - 1)
        );

        if (NULL == bufferList) {NSLog (@"*** malloc failure for allocating bufferList memory"); return;}

        // initialize the mNumberBuffers member
        bufferList->mNumberBuffers = channelCount;

        // initialize the mBuffers member to 0
        AudioBuffer emptyBuffer = {0};
        size_t arrayIndex;
        for (arrayIndex = 0; arrayIndex < channelCount; arrayIndex++) {
            bufferList->mBuffers[arrayIndex] = emptyBuffer;
        }

        // set up the AudioBuffer structs in the buffer list
        bufferList->mBuffers[0].mNumberChannels  = 1;
        bufferList->mBuffers[0].mDataByteSize    = totalFramesInFile * sizeof (AudioUnitSampleType);
        bufferList->mBuffers[0].mData            = soundStructArray[audioFile].audioDataLeft;

        if (2 == channelCount) {
            bufferList->mBuffers[1].mNumberChannels  = 1;
            bufferList->mBuffers[1].mDataByteSize    = totalFramesInFile * sizeof (AudioUnitSampleType);
            bufferList->mBuffers[1].mData            = soundStructArray[audioFile].audioDataRight;
        }

        // Perform a synchronous, sequential read of the audio data out of the file and
        //    into the soundStructArray[audioFile].audioDataLeft and (if stereo) .audioDataRight members.
        UInt32 numberOfPacketsToRead = (UInt32) totalFramesInFile;

        result = ExtAudioFileRead (
                     audioFileObject,
                     &numberOfPacketsToRead,
                     bufferList
                 );

        free (bufferList);

        if (noErr != result) {

            [self printErrorMessage: @"ExtAudioFileRead failure - " withStatus: result];

            // If reading from the file failed, then free the memory for the sound buffer.
            free (soundStructArray[audioFile].audioDataLeft);
            soundStructArray[audioFile].audioDataLeft = 0;

            if (2 == channelCount) {
                free (soundStructArray[audioFile].audioDataRight);
                soundStructArray[audioFile].audioDataRight = 0;
            }

            ExtAudioFileDispose (audioFileObject);            
            return;
        }

        NSLog (@"Finished reading file %i into memory", audioFile);

        // Set the sample index to zero, so that playback starts at the 
        //    beginning of the sound.
        soundStructArray[audioFile].sampleNumber = 0;

        // Dispose of the extended audio file object, which also
        //    closes the associated file.
        ExtAudioFileDispose (audioFileObject);
    }
}

Which part contains the array of audio samples which have to be reversed? Is it the AudioUnitSampleType?

bufferList->mBuffers[0].mData = soundStructArray[audioFile].audioDataLeft;

Note: audioDataLeft is defined as an AudioUnitSampleType, which is an SInt32 but not an array.

I found a clue in a Core Audio Mailing list:

Well, nothing to do with iPh*n* as far as I know (unless some audio API has been omitted -- I am not a member of that program). AFAIR, AudioFile.h and ExtendedAudioFile.h should provide you with what you need to read or write a caf and access its streams/channels. Basically, you want to read each channel/stream backwards, so, if you don't need properties of the audio file it is pretty straightforward once you have a handle on that channel's data, assuming it is not in a compressed format. Considering the number of formats a caf can represent, this could take a few more lines of code than you're thinking. Once you have a handle on uncompressed data, it should be about as easy as reversing a string. Then you would of course replace the file's data with the reversed data, or you could just feed the audio output (or wherever you're sending the reversed signal) reading whatever stream you have backwards.

This is what I tried, but when I assign my reversed buffer to the mData of both channels, I hear nothing:

AudioUnitSampleType *leftData = soundStructArray[audioFile].audioDataLeft;
AudioUnitSampleType *reversedData = (AudioUnitSampleType *) calloc (totalFramesInFile, sizeof (AudioUnitSampleType));
UInt64 j = 0;
for (UInt64 i = (totalFramesInFile - 1); i > -1; i--) {
    reversedData[j] = leftData[i];
    j++;
}

Answer 1

Typically, when an ASBD is being used, the fields describe the complete layout of the sample data in the buffers that are represented by this description - where typically those buffers are represented by an AudioBuffer that is contained in an AudioBufferList.

However, when an ASBD has the kAudioFormatFlagIsNonInterleaved flag, the AudioBufferList has a different structure and semantic. In this case, the ASBD fields will describe the format of ONE of the AudioBuffers that are contained in the list, AND each AudioBuffer in the list is determined to have a single (mono) channel of audio data. Then, the ASBD's mChannelsPerFrame will indicate the total number of AudioBuffers that are contained within the AudioBufferList - where each buffer contains one channel. This is used primarily with the AudioUnit (and AudioConverter) representation of this list - and won't be found in the AudioHardware usage of this structure.

Answer 2

I have worked on a sample app, which records what user says and plays them backwards. I have used CoreAudio to achieve this. Link to app code.

/* As each sample is 16-bits in size(2 bytes)(mono channel). You can load each sample at a time by copying it into a different buffer by starting at the end of the recording and reading backwards. When you get to the start of the data you have reversed the data and playing will be reversed. */

// set up output file
AudioFileID outputAudioFile;

AudioStreamBasicDescription myPCMFormat;
myPCMFormat.mSampleRate = 16000.00;
myPCMFormat.mFormatID = kAudioFormatLinearPCM ;
myPCMFormat.mFormatFlags =  kAudioFormatFlagsCanonical;
myPCMFormat.mChannelsPerFrame = 1;
myPCMFormat.mFramesPerPacket = 1;
myPCMFormat.mBitsPerChannel = 16;
myPCMFormat.mBytesPerPacket = 2;
myPCMFormat.mBytesPerFrame = 2;


AudioFileCreateWithURL((__bridge CFURLRef)self.flippedAudioUrl,
                       kAudioFileCAFType,
                       &myPCMFormat,
                       kAudioFileFlags_EraseFile,
                       &outputAudioFile);
// set up input file
AudioFileID inputAudioFile;
OSStatus theErr = noErr;
UInt64 fileDataSize = 0;

AudioStreamBasicDescription theFileFormat;
UInt32 thePropertySize = sizeof(theFileFormat);

theErr = AudioFileOpenURL((__bridge CFURLRef)self.recordedAudioUrl, kAudioFileReadPermission, 0, &inputAudioFile);

thePropertySize = sizeof(fileDataSize);
theErr = AudioFileGetProperty(inputAudioFile, kAudioFilePropertyAudioDataByteCount, &thePropertySize, &fileDataSize);

UInt32 dataSize = fileDataSize;
void* theData = malloc(dataSize);

//Read data into buffer
UInt32 readPoint  = dataSize;
UInt32 writePoint = 0;
while( readPoint > 0 )
{
    UInt32 bytesToRead = 2;

    AudioFileReadBytes( inputAudioFile, false, readPoint, &bytesToRead, theData );
    AudioFileWriteBytes( outputAudioFile, false, writePoint, &bytesToRead, theData );

    writePoint += 2;
    readPoint -= 2;
}

free(theData);
AudioFileClose(inputAudioFile);
AudioFileClose(outputAudioFile);

Hope this helps.

Answer 3

You do not have to allocate a separate buffer to store the reversed data, this can take a fair bit of CPU, depending on the length of sound. To play a sound backwards, just make the sampleNumber counter start at totalFramesInFile - 1.

You can modify MixerHost like this, to achieve the desired effect.

Replace soundStructArray[audioFile].sampleNumber = 0; with soundStructArray[audioFile].sampleNumber = totalFramesInFile - 1;

Make sampleNumber SInt32 instead of UInt32.

Replace the loop which you write the samples out with this.

for (UInt32 frameNumber = 0; frameNumber < inNumberFrames; ++frameNumber) {
    outSamplesChannelLeft[frameNumber]                 = dataInLeft[sampleNumber];
    if (isStereo) outSamplesChannelRight[frameNumber]  = dataInRight[sampleNumber];

    if (--sampleNumber < 0) sampleNumber = frameTotalForSound - 1;
}

This effectively makes it play backwards. Mmmm. It's been a while since I've heard the MixerHost music. I must admit I find it to be quite pleasing.