Android audio programming nightmare - soundpool, audiotrack arrghh?

Question

I've built a simple music sequencer Android app that plays multiple audio files.

Originally I was using SoundPool to play mp3 files and it worked perfectly on 2.3.4 with an old HTC Droid Incredible. Then I tested it on a Galaxy Nexus running 4.3 and the performance was horrendous. The audio timing all over the place and there were glitches/clicks/pops.

So I spent several days making a player using AudioTrack including an mp3 decoder and got it working perfectly on both the Galaxy and the HTC. Now I've just tested it on a Nexus 4 (running 4.3) and the performance is terrible - the timing is all over the place. SoundPool even offers better performance on this device.

I'm really frustrated and don't know what to do to finish my app so I would really appreciate if someone could help me out. I've put some code samples of my audio player below. I've tried everything I can think of including changing the buffer size, using AudioTrack.MODE_STATIC etc. The new Google devices have low latency audio so it's very strange how everything works way better on my old droid!

Thanks in advance

/**
* Play note
*/
public void playNote(String note, float vol)
{
    PlayThread oldThread = threadMap.get(note);
    if(oldThread != null) {
        //Cancel timer
        if(oldThread.timer != null) {
            oldThread.timer.cancel();
            oldThread.timer.purge();
            oldThread.timer = null;
        }
        //Stop
        oldThread.requestStop();
        threadMap.remove(note);
    }

    //Play if within Polyphony
    if(threadMap.size() < POLYPHONY) {
        PlayThread thread = new PlayThread(note, vol);
        thread.start();
        threadMap.put(note, thread);
    }       
}


/**
* Stop note
*/
public void stopNote(String note, int fadeDurationInMs)
{   
    PlayThread thread = threadMap.get(note);
    if(thread != null) {
        thread.fadeOut(fadeDurationInMs);
        threadMap.remove(note);
    }
}


/**
* Stop all
*/
public void stopAllPlaying(int fadeDurationInMs)
{
    for(PlayThread thread : threadMap.values()) {
        if(thread != null) {
            thread.fadeOut(fadeDurationInMs);
        }
    }
    threadMap.clear();
}


/**
* PlayThread
*/
private class PlayThread extends Thread
{
    String note;
    float vol;
    float fadeVol;
    boolean stop;
    AudioTrack audioTrack;
    Timer timer;

    /**
    * Constructor
    */
    public PlayThread(String note, float vol)
    {
        super();
        this.note = note;
        this.vol = vol;
        this.fadeVol = vol;
    }

    /**
    * Run
    */
    public void run()
    {
        try {
            android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);

            //Create buffer
            int bufferSize = AudioTrack.getMinBufferSize(SAMPLE_RATE, CHANNELS, AudioFormat.ENCODING_PCM_16BIT);
            Log.v(Constants.TAG, "min buffersize = " + bufferSize);

            bufferSize = bufferSize * 2;

            byte[] buffer = new byte[bufferSize];

            //AudioTrack
            audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, SAMPLE_RATE, CHANNELS, AudioFormat.ENCODING_PCM_16BIT, bufferSize, AudioTrack.MODE_STREAM);
            audioTrack.setStereoVolume(vol, vol);
            audioTrack.play();

            //Get byte data
            byte[] byteData = sampleMap.get(note);

            //Convert to input stream
            InputStream input = new ByteArrayInputStream(byteData);

            //Write to audioTrack
            int bytesRead = 0;
            while(!stop && (bytesRead = input.read(buffer)) != -1) {
                    audioTrack.write(buffer, 0, bytesRead);
            }

            //When finished...
            audioTrack.stop();
            audioTrack.release();
            input.close();
            killThread(this);
        }
        catch(Exception e) {}
    }

    /**
    * Set volume
    */
    private synchronized void setVol(float newVol)
    {
        audioTrack.setStereoVolume(newVol, newVol);
    }

    /**
    * Update volume
    */
    private synchronized void lowerVol()
    {
        fadeVol -= 0.01;
        if(fadeVol < 0) vol = 0;
        audioTrack.setStereoVolume(fadeVol, fadeVol);
    }

    /**
    * Fade out
    */
    public synchronized void fadeOut(int fadeDurationInMs)
    {
        //Start decreasing volume
        if(fadeDurationInMs > 0) {
            timer = new Timer(true);
            TimerTask timerTask = new TimerTask()
            {
                @Override
                public void run()
                {
                    //If thread killed while running
                    try {
                        //Lower volume
                        lowerVol();
                    }
                    catch (Exception e) {}

                    //Stop when volume reaches 0
                    if(fadeVol <= 0) {
                        if(timer != null) {
                            timer.cancel();
                            timer.purge();
                        }
                        stop = true;
                    }
                }
            };

            //Calculate delay, set to 1 if zero
            int delay = (int) (fadeDurationInMs / (vol * 100)); 
            if(delay == 0) delay = 1;

            timer.schedule(timerTask, delay, delay);
        }
    }

    /**
    * Request stop
    */
    public synchronized void requestStop()
    {
        //Stop click/pop when stopping sample
        setVol(0.01f);
        setVol(0.005f);
        stop = true;
    }

    /**
    * Kill Thread
    */
    private synchronized void killThread(Thread theThread)
    {
        if(theThread != null) {
            theThread = null;
        }
    }

}

Answer 1

Like user harikris suggests, I would highly recommend you move all your audio playback and processing code to Android NDK using the OpenSL ES library for the best performance.

As I understand, the AudioTrack API is built on top of OpenSL ES Buffer Queue Audio Player. So you could probably improve performance by working directly with the NDK, writing C code that is called from your Java/Android layer to work with the sound.

The native-audio example mentioned above contains code that will show you how to play a sound file directly from a URI. In my experience, the results from this method are better than AudioTrack in Static Mode.

Soundpool is generally reserved for very short sounds that can be played from memory and its not a scalable solution for your sequencer especially if you introduce large files.

Here are some links that have helped me out with my applications: -General Information on OpenSL ES for Android: http://mobilepearls.com/labs/native-android-api/opensles/

-An Android audio blog with some great example code: http://audioprograming.wordpress.com

Edit: The old mobile pearl link appears to be down. Here's a working one:http://mobilepearls.com/labs/native-android-api/ndk/docs/opensles/index.html

Answer 2

In the High Performance Audio session at Google I/O 2013 (introduced by Ian Ni-Lewis who commented on the original post. I'm surprised he didn't pick up on this), they talk about the Nexus 4. In the video of the presentation, jump to 27:25.

Unlike many other devices that use a native 44.1kHz sample rate, the Nexus 4 uses 48kHz. If you are submitting data at 44.1kHz, it will have to go through the resampler which is on the slow path. Another difference is that the buffer size on the Nexus 4 is 240 frames which can cause a regular jitter in your callback. This is all explained in the linked video.

Answer 3

I would recommend you do all your audio capture and time sensitive processing in native code and use JNI to interact between the SDK and NDK components.

You can find native-audio project example code in the Android NDK distro that show you how to do audio in C/C++