Issue in passing Egl configured surface to native and push the data

Question

First, thanks to fadden for your wonderful examples.

I tried following this ContinuousCapture.java example, and produced the below program.

1)

I am trying to display the image into TextureView in the native layer by getting its ANativeWwindow reference, and using ANative lock and unlockpost methods to acquire the BufferQueue and to fill the data.

Ie:

ANativeWindow_lock(*window, &buffer, NULL)

ANativeWindow_unlockAndPost(*window);

2)

At the same time i want to retrieve the data from this Surface and pass it to the encoder. Or, display it into another surface.

As a first step, I have created the below Class which initializes the EglCore into different thread, and tries to Configure user given surface in EglContext. So, far so good. But when i try to copy the data into the buffer as i mentioned earlier by lock & unlockAndPost methods, i recive the below errors.

 E/BufferQueueProducer: [unnamed-7679-0] connect(P): already connected (cur=1 req=2)
E/BufferQueueProducer: [unnamed-7679-0] connect(P): already connected (cur=1 req=2)
E/BufferQueueProducer: [unnamed-7679-0] connect(P): already connected (cur=1 req=2)

Question: Is it the right approach?. OR did i overlook something?

package com.super.dump

import android.graphics.Rect;
import android.graphics.SurfaceTexture;
import android.hardware.Camera;
import android.opengl.EGLSurface;
import android.os.Environment;
import android.os.Handler;
import android.os.Looper;
import android.os.Message;
import android.util.Log;
import android.util.SparseArray;
import android.view.Surface;

import java.io.File;
import java.io.IOException;



import android.graphics.SurfaceTexture;
import android.opengl.EGL14;
import android.opengl.EGLSurface;
import android.opengl.GLES20;
import android.os.Environment;
import android.util.Log;
import android.util.SparseArray;
import android.view.Surface;
import android.view.View;
import android.widget.TextView;

import java.io.File;
import java.io.IOException;
import java.lang.ref.WeakReference;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;

 public class SuperDump {

    RenderThread rT;
    Surface userProvidedSurface;


    public SuperDump() {
        userProvidedSurface = null;
        rT = null;
    }


    public void init(Surface userSurface)
    {
        if ( userSurface != null) {
            userProvidedSurface = userSurface;
            rT = new RenderThread(userProvidedSurface);
            rT.start();
            rT.waitUntilRendererReady();
        }
    }


    private class RenderThread extends Thread {

        public String TAG = "RenderThread";
        RenderHandler mHandler;
        private Object mSyncForRenderAvailability = new Object();
        boolean mIsRendererReady = false;

        private EglCore mEglCore;
        private Surface mSurfaceUser;
        private WindowSurface mSurfaceWindowUser;


        public RenderThread() {

        }

        public RenderThread(Surface userSurface) {
            mSurfaceUser = userSurface;
        }



        @Override
        public void run() {
            Looper.prepare();
            mHandler = new RenderHandler(this);

            mEglCore = new EglCore(null, EglCore.FLAG_RECORDABLE);
            mSurfaceWindowUser = new WindowSurface(mEglCore, mSurfaceUser, false);

            synchronized (mSyncForRenderAvailability) {

                mIsRendererReady = true;
                mSyncForRenderAvailability.notifyAll();
            }

            Looper.loop();
            Log.d (TAG, "End of RenderThread..");
        }

        public RenderHandler getHandler() {
            return mHandler;
        }


        public void waitUntilRendererReady()
        {
            synchronized (mSyncForRenderAvailability) {
                while(!mIsRendererReady) {
                    try {
                        mSyncForRenderAvailability.wait();
                    } catch (InterruptedException e) {
                        Log.d (TAG,  "Wait interrupted..");
                    }
                }
            }
        }

}  // RenderThread





   private static class RenderHandler extends Handler {

       public String TAG = "RenderHandler";
       private static final int MSG_RENDER_QUIT = 1;

        private WeakReference<RenderThread> mWeakRenderThread;

        public RenderHandler(RenderThread rT)
        {
            mWeakRenderThread = new WeakReference<RenderThread>(rT);
        }

       public void stopMe() {
           sendMessage(obtainMessage(MSG_RENDER_QUIT));
       }

        @Override
       public void handleMessage(Message msg) {

           Log.d (TAG, "Inside handleMessage..");

            switch(msg.what) {
                case MSG_RENDER_QUIT:
                    Looper.getMainLooper().quit();  // detaching from thread.
            }
       }




   }  // RenderHandler Class.
}; //SuperDump class

Kindly help me.

Answer 1

At the same time i want to retrieve the data from this Surface and pass it to the encoder

You can't retrieve data from a Surface. Surfaces are the producer side of a producer-consumer pair.

You're getting the "already connected" errors because you're attempting to attach a second producer to the Surface.

(If you want to pick nits, BufferQueue will re-use buffers without clearing them, so the producer can often see the data from a frame or two back. But since the producer created those frames in the first place, there's not much value in pulling them back out. And there is no guarantee that BufferQueue won't give you an empty buffer, or present them out of order.)

The good news is that, for a SurfaceTexture, the consumer is in-process and the data is accessible as a GLES "external" texture. The TextureView class provides a convenient getBitmap() method that renders the most recent frame into a Bitmap, allowing direct access to the pixels. (You can do something similar by rendering the texture to an off-screen pbuffer and reading the pixels out with glReadPixels().)

So to implement the solution you want, the easiest thing would be to just use the TextureView call to get a Bitmap.

More information about Surfaces can be found in the graphics architecture doc.