MediaCodec with Surface Input: Recording in background

Question

I'm working on a video encoding application which I want to prevent from stopping when the hosting Activity enters the background, or the screen cycles off/on.

The architecture of my encoder is derived from the excellent CameraToMpegTest example, with the addition of displaying camera frames to a GLSurfaceView (see Github links below). I'm currently performing background recording with a two-state solution:

• When the hosting Activity is in the foreground, encode one video frame on each call to the GLSurfaceView.Renderer's onDrawFrame. This allows me access to the GLSurfaceView's EGL state in bursts so as not to block other events queued to the renderer thread.

• When the hosting Activity enters the background, halt the onDrawFrame encoding and encode frames on another background thread within a loop. This mode is identical to the CameraToMpegTest example.

However if the screen is powered off the GLSurfaceView's EGLContext is lost and a new call to onSurfaceCreated occurs. In this case we have to re-create the EGL window surface connected to MediaCodec's input Surface. Unfortunately this 2nd call to eglCreateWindowSurface produces:

E/libEGL(18839): EGLNativeWindowType 0x7a931098 already connected to another API

Prior to calling, I release all EGL resources connected to the Android Surface.

Is there a way to swap the EGLSurface connected to MediaCodec's input Surface?

The complete source of my test application is on Github. Main Activity.

Update I applied the lessons learned here into a video sdk for Android based on the MediaCodec & MediaMuxer classes. Hope it helps!

Answer 1

Background first...

When you call eglCreateWindowSurface(), the Android EGL wrapper calls native_window_api_connect() on the Surface you passed in. This eventually turns into a BufferQueue producer connect call, which means that this EGL surface is now the sole source of graphics buffers for the Surface.

The EGL surface stays connected to the Surface until the EGL surface is destroyed. When it is, the surface destructor calls native_window_api_disconnect() to disconnect the EGL surface from the BufferQueue. The EGL surface is reference-counted, with the refcount incremented when the surface is passed to eglMakeCurrent() so to be destroyed two things must happen:

1. eglDestroySurface() must be called
2. the EGL surface must not be "current" in any thread

The second item requires calling eglMakeCurrent() with another EGL surface (or EGL_NO_SURFACE), or calling eglReleaseThread(), on any thread that had previously used the surface. One quick way to confirm that this is done is to add logging before calls to eglMakeCurrent() when the surface is made current and un-current, and compare the thread IDs by viewing the logcat output with adb logcat -v threadtime. It may also be useful to use EGL queries like eglGetCurrentSurface(EGL_DRAW) to confirm that you're doing the un-current in the thread that has made the surface current.

If the EGL surface isn't destroyed, it won't disconnect from the Surface, and attempts to connect a new producer (by calling eglCreateWindowSurface with a new EGL surface) will be rejected with the "already connected" message.

Update: My implementation is now available in the Grafika test project. If you install this, select "Show + capture camera", start recording, toggle the power, and then stop recording, you should have a complete movie with a long pause in the middle. You can back out, select "Play video", and choose "camera-test.mp4" to view it.