MediaPlayer skips forward about 6 seconds on rotation

Question

I have a MediaPlayer in a Fragment which retains its instance on configuration changes. The player is playing a video loaded from my assets directory. I have the scenario set up with the goal of reproducing the YouTube app playback where the audio keeps playing during the configuration changes and the display is detached and reattached to the media player.

When I start the playback and rotate the device, the position jumps forward about 6 seconds and (necessarily) the audio cuts out when this happens. Afterwards, the playback continues normally. I have no idea what could be causing this to happen.

As requested, here is the code:

public class MainFragment extends Fragment implements SurfaceHolder.Callback, MediaController.MediaPlayerControl {

    private static final String TAG = MainFragment.class.getSimpleName();

    AssetFileDescriptor mVideoFd;

    SurfaceView mSurfaceView;
    MediaPlayer mMediaPlayer;
    MediaController mMediaController;
    boolean mPrepared;
    boolean mShouldResumePlayback;
    int mBufferingPercent;
    SurfaceHolder mSurfaceHolder;

    @Override
    public void onInflate(Activity activity, AttributeSet attrs, Bundle savedInstanceState) {
        super.onInflate(activity, attrs, savedInstanceState);
        final String assetFileName = "test-video.mp4";
        try {
            mVideoFd = activity.getAssets().openFd(assetFileName);
        } catch (IOException ioe) {
            Log.e(TAG, "Can't open file " + assetFileName + "!");
        }
    }

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setRetainInstance(true);

        // initialize the media player
        mMediaPlayer = new MediaPlayer();
        try {
            mMediaPlayer.setDataSource(mVideoFd.getFileDescriptor(), mVideoFd.getStartOffset(), mVideoFd.getLength());
        } catch (IOException ioe) {
            Log.e(TAG, "Unable to read video file when setting data source.");
            throw new RuntimeException("Can't read assets file!");
        }

        mMediaPlayer.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
            @Override
            public void onPrepared(MediaPlayer mp) {
                mPrepared = true;
            }
        });

        mMediaPlayer.setOnBufferingUpdateListener(new MediaPlayer.OnBufferingUpdateListener() {
            @Override
            public void onBufferingUpdate(MediaPlayer mp, int percent) {
                mBufferingPercent = percent;
            }
        });

        mMediaPlayer.prepareAsync();
    }

    @Override
    public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
        super.onCreateView(inflater, container, savedInstanceState);
        View view = inflater.inflate(R.layout.fragment_main, container, false);
        mSurfaceView = (SurfaceView) view.findViewById(R.id.surface);
        mSurfaceView.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                mMediaController.show();
            }
        });

        mSurfaceHolder = mSurfaceView.getHolder();
        if (mSurfaceHolder == null) {
            throw new RuntimeException("SufraceView's holder is null");
        }
        mSurfaceHolder.addCallback(this);
        return view;
    }

    @Override
    public void onViewCreated(View view, Bundle savedInstanceState) {
        super.onViewCreated(view, savedInstanceState);
        mMediaController = new MediaController(getActivity());
        mMediaController.setEnabled(false);
        mMediaController.setMediaPlayer(this);
        mMediaController.setAnchorView(view);
    }

    @Override
    public void onResume() {
        super.onResume();
        if (mShouldResumePlayback) {
            start();
        } else {
            mSurfaceView.post(new Runnable() {
                @Override
                public void run() {
                    mMediaController.show();
                }
            });
        }
    }

    @Override
    public void surfaceCreated(SurfaceHolder holder) {
        mMediaPlayer.setDisplay(mSurfaceHolder);
        mMediaController.setEnabled(true);
    }

    @Override
    public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
        // nothing
    }

    @Override
    public void surfaceDestroyed(SurfaceHolder holder) {
        mMediaPlayer.setDisplay(null);
    }

    @Override
    public void onPause() {
        if (mMediaPlayer.isPlaying() && !getActivity().isChangingConfigurations()) {
            pause();
            mShouldResumePlayback = true;
        }
        super.onPause();
    }

    @Override
    public void onSaveInstanceState(Bundle outState) {
        super.onSaveInstanceState(outState);

    }

    @Override
    public void onDestroyView() {
        mMediaController.setAnchorView(null);
        mMediaController = null;
        mMediaPlayer.setDisplay(null);
        mSurfaceHolder.removeCallback(this);
        mSurfaceHolder = null;
        mSurfaceView = null;
        super.onDestroyView();
    }

    @Override
    public void onDestroy() {
        mMediaPlayer.release();
        mMediaPlayer = null;
        try {
            mVideoFd.close();
        } catch (IOException ioe) {
            Log.e(TAG, "Can't close asset file..", ioe);
        }
        mVideoFd = null;
        super.onDestroy();
    }

    // MediaControler methods:
    @Override
    public void start() {
        mMediaPlayer.start();
    }

    @Override
    public void pause() {
        mMediaPlayer.pause();
    }

    @Override
    public int getDuration() {
        return mMediaPlayer.getDuration();
    }

    @Override
    public int getCurrentPosition() {
        return mMediaPlayer.getCurrentPosition();
    }

    @Override
    public void seekTo(int pos) {
        mMediaPlayer.seekTo(pos);
    }

    @Override
    public boolean isPlaying() {
        return mMediaPlayer.isPlaying();
    }

    @Override
    public int getBufferPercentage() {
        return mBufferingPercent;
    }

    @Override
    public boolean canPause() {
        return true;
    }

    @Override
    public boolean canSeekBackward() {
        return true;
    }

    @Override
    public boolean canSeekForward() {
        return true;
    }

    @Override
    public int getAudioSessionId() {
        return mMediaPlayer.getAudioSessionId();
    }
}

The if block in the onPause method is not being hit.

Update:

After doing a bit more debugging, removing the interaction with the SurfaceHolder causes the problem to go away. In other words, if I don't setDisplay on the MediaPlayer the audio will work fine during the configuration change: no pause, no skip. It would seem there is some timing issue with setting the display on the MediaPlayer that is confusing the player.

Additionally, I have found that you must hide() the MediaController before you remove it during the configuration change. This improves stability but does not fix the skipping issue.

Another update:

If you care, the Android media stack looks like this:

MediaPlayer.java 
 -> android_media_MediaPlayer.cpp 
 -> MediaPlayer.cpp
 -> IMediaPlayer.cpp 
 -> MediaPlayerService.cpp 
 -> BnMediaPlayerService.cpp 
 -> IMediaPlayerService.cpp 
 -> *ConcreteMediaPlayer*
 -> *BaseMediaPlayer* (Stagefright, NuPlayerDriver, Midi, etc) 
 -> *real MediaPlayerProxy* (AwesomePlayer, NuPlayer, etc) 
 -> *RealMediaPlayer* (AwesomePlayerSource, NuPlayerDecoder, etc) 
  -> Codec
  -> HW/SW decoder

Upon examining AwesomePlayer, it appears this awesome player takes the liberty of pausing itself for you when you setSurface():

status_t AwesomePlayer::setNativeWindow_l(const sp<ANativeWindow> &native) {
    mNativeWindow = native;

    if (mVideoSource == NULL) {
        return OK;
    }

    ALOGV("attempting to reconfigure to use new surface");

    bool wasPlaying = (mFlags & PLAYING) != 0;

    pause_l();
    mVideoRenderer.clear();

    shutdownVideoDecoder_l();

    status_t err = initVideoDecoder();

    if (err != OK) {
        ALOGE("failed to reinstantiate video decoder after surface change.");
        return err;
    }

    if (mLastVideoTimeUs >= 0) {
        mSeeking = SEEK;
        mSeekTimeUs = mLastVideoTimeUs;
        modifyFlags((AT_EOS | AUDIO_AT_EOS | VIDEO_AT_EOS), CLEAR);
    }

    if (wasPlaying) {
        play_l();
    }

    return OK;
}

This reveals that setting the surface will cause the player to destroy whatever surface was previously being used as well as the video decoder along with it. While setting a surface to null should not cause the audio to stop, setting it to a new surface requires the video decoder to be reinitialized and the player to seek to the current location in the video. By convention, seeking will never take you further than you request, that is, if you overshoot a keyframe when seeking, you should land on the frame you overshot (as opposed to the next one).

My hypothesis, then, is that the Android MediaPlayer does not honor this convention and jumps forward to the next keyframe when seeking. This, coupled with a video source that has sparse keyframes, could explain the jumping I am experiencing. I have not looked at AwesomePlayer's implementation of seek, though. It was mentioned to me that jumping to the next keyframe is something that needs to happen if your MediaPlayer is developed with streaming in mind since the stream can be discarded as soon as it has been consumed. Point being, it might not be that far fetch to think the MediaPlayer would choose to jump forward as opposed to backwards.

Final Update:

While I still don't know why the playback skips when attaching a new Surface as the display for a MediaPlayer, thanks to the accepted answer, I have gotten the playback to be seamless during rotation.

Answer 1

Thanks to natez0r's answer, I have managed to get the setup described working. However, I use a slightly different method. I'll detail it here for reference.

I have one Fragment which I flag to be retained on configuration changes. This fragment handles both the media playback (MediaPlayer), and the standard TextureView (which provides the SurfaceTexture where the video buffer gets dumped). I initialize the media playback only once my Activity has finished onResume() and once the SurfaceTexture is available. Instead of subclassing TextureView, I simply call setSurfaceTexture (since it's public) in my fragment once I receive a reference to the SurfaceTexture. The only two things retained when a configuration change happens are the MediaPlayer reference, and the SurfaceTexture reference.

I've uploaded the source of my sample project to Github. Feel free to take a look!

Answer 2

I know this question is a tad old now, but I was able to get this working in my app without the skipping. The issue is the surface getting destroyed (killing whatever buffer it had in it). This may not solve all your issues because it targets API 16, but you can manage your own SurfaceTexture inside your custom TextureView where the video is drawn:

private SurfaceTexture mTexture;

    private TextureView.SurfaceTextureListener mSHCallback =
            new TextureView.SurfaceTextureListener() {
                @Override
                public void onSurfaceTextureAvailable(SurfaceTexture surface, int width,
                        int height) {
                    mTexture = surface;
                    mPlayer.setSurface(new Surface(mTexture));
                }

                @Override
                public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width,
                        int height) {
                    mTexture = surface;
                }

                @Override
                public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
                    mTexture = surface;
                    return false;
                }

                @Override
                public void onSurfaceTextureUpdated(SurfaceTexture surface) {
                    mTexture = surface;
                }
            };

the key is returning false in onSurfaceTextureDestroyed and holding onto mTexture. When the view gets re-attached to the window you can set the surfaceTexture:

    @Override
    protected void onAttachedToWindow() {
        super.onAttachedToWindow();
        if (mTexture != null) {
            setSurfaceTexture(mTexture);
        }
    }

This allows my view to continue playing video from EXACTLY where it left off.