Camera2 video recording without preview on Android: mp4 output file not fully playable

Question

I am trying to record video from the back camera (the one that faces the face) on my Samsung Galaxy S6 (which supports 1920x1080 at about 30 fps). I do not want to have to

Answer 1

My team encountered a similar problem when we were developing a plugin based on the Camera2 API, but it only affected a Samsung Galaxy S7 (we also have an S6 for testing that didn't exhibit this behaviour).

The issue appeared to be caused by a bug in Samsung's camera firmware and was triggered when the device came out of Deep Sleep (the ultra-low power mode in Android 6.0 Marshmallow). After resuming from Deep Sleep, the first frame of any video captured and encoded using the Camera2 MediaRecorder has an extraordinarily long frame duration - sometimes as long as or longer than the total duration of the video itself.

Consequently, when playing back, the first frame is displayed for this long duration while audio continues to play. Once the first frame has finished displaying, the rest of the frames play back as normal.

We found other people with a similar problem discussing the issue on GitHub

The issue is a deep sleep problem on some devices running Marshmallow. It appears to be CPU related as an S7 on Verizon doesn't have the issue, but an S7 on AT&T does have the issue. I've seen this on an S6 Verizon phone when it updated to Marshmallow.

In order to replicate, reboot a device while connected to USB. Run the sample. All should be ok. Then, disconnect the device, let it go into deep sleep (screen off, no movement for 5? minutes), and try again. The issue will appear once the device has gone into deep sleep.

We ended up using cybaker's proposed workaround; that is, when the video file is created, inspect the duration of the first frame of the video. If it appears to be incorrect, re-encode the video with sensible frame durations:

DataSource channel = new FileDataSourceImpl(rawFile);
IsoFile isoFile = new IsoFile(channel);

List<TrackBox> trackBoxes = isoFile.getMovieBox().getBoxes(TrackBox.class);
boolean sampleError = false;
for (TrackBox trackBox : trackBoxes) {
    TimeToSampleBox.Entry firstEntry = trackBox.getMediaBox().getMediaInformationBox().getSampleTableBox().getTimeToSampleBox().getEntries().get(0);

    // Detect if first sample is a problem and fix it in isoFile
    // This is a hack. The audio deltas are 1024 for my files, and video deltas about 3000
    // 10000 seems sufficient since for 30 fps the normal delta is about 3000
    if(firstEntry.getDelta() > 10000) {
        sampleError = true;
        firstEntry.setDelta(3000);
    }
}

if(sampleError) {
    Movie movie = new Movie();
    for (TrackBox trackBox : trackBoxes) {
            movie.addTrack(new Mp4TrackImpl(channel.toString() + "[" + trackBox.getTrackHeaderBox().getTrackId() + "]" , trackBox));
    }
    movie.setMatrix(isoFile.getMovieBox().getMovieHeaderBox().getMatrix());
    Container out = new DefaultMp4Builder().build(movie);

    //delete file first!
    FileChannel fc = new RandomAccessFile(rawFile.getName(), "rw").getChannel();
    out.writeContainer(fc);
    fc.close();
    Log.d(TAG, "Finished correcting raw video");
}

Hope this points you in the right direction!

Answer 2

Note the code above posted by harper requires these dependencies:

compile 'com.googlecode.mp4parser:isoparser:1.1.22'