android: SurfaceTexure, camera frame wait time out

Question

I\'m trying to use MediaCodec and MediaMux, and I meet some trouble.

Here is the errors from the logcat:

12-13 11:59:58.238: E/AndroidRuntime(23218):         


        

Answer 1

As Florian correctly explained, the issue is that your code is running on a thread that has a looper. You need to make sure that the code is running on a thread that does not have a looper.

The way I solved it was by modifying the setup() method in OutputSurface and ensuring that the setOnFrameListener() is attached to another Handler Thread.

Here is the code for the same:

class OutputSurface implements SurfaceTexture.OnFrameAvailableListener {
    private static final String TAG = "OutputSurface";
    private static final boolean VERBOSE = false;
    private EGLDisplay mEGLDisplay = EGL14.EGL_NO_DISPLAY;
    private EGLContext mEGLContext = EGL14.EGL_NO_CONTEXT;
    private EGLSurface mEGLSurface = EGL14.EGL_NO_SURFACE;
    private SurfaceTexture mSurfaceTexture;
    private Surface mSurface;
    private Object mFrameSyncObject = new Object();     
    private boolean mFrameAvailable;
    private TextureRender mTextureRender;

    private HandlerThread mHandlerThread;
    private Handler mHandler;

    public OutputSurface(int width, int height) {
        if (width <= 0 || height <= 0) {
            throw new IllegalArgumentException();
        }
        eglSetup(width, height);
        makeCurrent();
        setup();
    }

    public OutputSurface() {
        setup();
    }

    private void setup() {
        mTextureRender = new TextureRender();
        mTextureRender.surfaceCreated();

        mHandlerThread = new HandlerThread("callback-thread");
        mHandlerThread.start();
        mHandler = new Handler(mHandlerThread.getLooper());

        // Even if we don't access the SurfaceTexture after the constructor returns, we
        // still need to keep a reference to it.  The Surface doesn't retain a reference
        // at the Java level, so if we don't either then the object can get GCed, which
        // causes the native finalizer to run.
        if (VERBOSE) Log.d(TAG, "textureID=" + mTextureRender.getTextureId());
        mSurfaceTexture = new SurfaceTexture(mTextureRender.getTextureId());
        // This doesn't work if OutputSurface is created on the thread that CTS started for
        // these test cases.
        //
        // The CTS-created thread has a Looper, and the SurfaceTexture constructor will
        // create a Handler that uses it.  The "frame available" message is delivered
        // there, but since we're not a Looper-based thread we'll never see it.  For
        // this to do anything useful, OutputSurface must be created on a thread without
        // a Looper, so that SurfaceTexture uses the main application Looper instead.
        //
        // Java language note: passing "this" out of a constructor is generally unwise,
        // but we should be able to get away with it here.

        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
            mSurfaceTexture.setOnFrameAvailableListener(this, mHandler);
        } else {
            mSurfaceTexture.setOnFrameAvailableListener(this);
        }

        mSurface = new Surface(mSurfaceTexture);
    }
}

The rest of the OutputSurface class can remain the same.

Answer 2

I encountered this issue as well. The reason therefore is that your code is running on a thread that has a looper. You have to make sure that the code is running on a thread that does not have a looper. If it does, SurfaceTexture.OnFrameAvailableListener will deliver the "frame available" message to the waiting thread, rather than sending the Message to the Handler on the main thread, and you'll get stuck.

Bigflake's examples provide you with a detailed description on that:

/**
 * Wraps testEditVideo, running it in a new thread.  Required because of the way
 * SurfaceTexture.OnFrameAvailableListener works when the current thread has a Looper
 * configured.
 */
private static class VideoEditWrapper implements Runnable {
    private Throwable mThrowable;
    private DecodeEditEncodeTest mTest;
    private VideoEditWrapper(DecodeEditEncodeTest test) {
        mTest = test;
    }
    @Override
    public void run() {
        try {
            mTest.videoEditTest();
        } catch (Throwable th) {
            mThrowable = th;
        }
    }
    /** Entry point. */
    public static void runTest(DecodeEditEncodeTest obj) throws Throwable {
        VideoEditWrapper wrapper = new VideoEditWrapper(obj);
        Thread th = new Thread(wrapper, "codec test");
        th.start();
        th.join();
        if (wrapper.mThrowable != null) {
            throw wrapper.mThrowable;
        }
    }
}