replacing glReadPixels with EGL_KHR_image_base for faster pixel copy

Question

I am trying to use EGL_KHR_image_base in an android native process in order to replace glReadPixels because it is to slow ( 220ms for 1280x800 RGBA ).

This is what I have so far, but my it produces an empty buffer ( only zeros )

uint8_t *ptr;
GLuint mTexture;
status_t error;

GraphicBufferAlloc* mGraphicBufferAlloc  = new GraphicBufferAlloc();    
sp<GraphicBuffer> window = mGraphicBufferAlloc->createGraphicBuffer(width, height, PIXEL_FORMAT_RGBA_8888, GraphicBuffer::USAGE_SW_READ_OFTEN | GraphicBuffer::USAGE_HW_TEXTURE,&error);
EGLClientBuffer buffer = (EGLClientBuffer)window->getNativeBuffer();
EGLint eglImageAttributes[] = {EGL_WIDTH, width, EGL_HEIGHT, height, EGL_MATCH_FORMAT_KHR,  EGL_FORMAT_RGBA_8888_KHR, EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE};
EGLImageKHR image = eglCreateImageKHR(eglGetCurrentDisplay(), EGL_NO_CONTEXT, EGL_NATIVE_BUFFER_ANDROID,buffer, eglImageAttributes);

glGenTextures(1, &mTexture);    
glBindTexture(GL_TEXTURE_2D, mTexture);
glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, image);
window->lock(GraphicBuffer::USAGE_SW_READ_OFTEN, (void**)&ptr); 
memcpy(texture, ptr, width * height * 4);
window->unlock();

What am I doing wrong ?

Answer 1

You're creating an empty buffer and then reading the contents out of it. Walking through the code:

GraphicBufferAlloc* mGraphicBufferAlloc = new GraphicBufferAlloc();
sp<GraphicBuffer> window = mGraphicBufferAlloc->createGraphicBuffer(width, height, PIXEL_FORMAT_RGBA_8888, GraphicBuffer::USAGE_SW_READ_OFTEN | GraphicBuffer::USAGE_HW_TEXTURE,&error);

This creates a new GraphicBuffer (sometimes referred to as a "gralloc buffer"), with the specified dimensions and pixel format. The usage flags allow it to be used as a texture or read from software, which is what you want.

EGLClientBuffer buffer = (EGLClientBuffer)window->getNativeBuffer();
EGLint eglImageAttributes[] = {EGL_WIDTH, width, EGL_HEIGHT, height, EGL_MATCH_FORMAT_KHR,  EGL_FORMAT_RGBA_8888_KHR, EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE};
EGLImageKHR image = eglCreateImageKHR(eglGetCurrentDisplay(), EGL_NO_CONTEXT, EGL_NATIVE_BUFFER_ANDROID,buffer, eglImageAttributes);

This takes the ANativeWindow object (which is a queue of buffers under the hood) and attaches an EGLImage "handle" to it.

glGenTextures(1, &mTexture);    
glBindTexture(GL_TEXTURE_2D, mTexture);
glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, image);

This creates a new texture object, and attaches the EGLImage to it. So now the ANativeWindow can be used as a texture object.

window->lock(GraphicBuffer::USAGE_SW_READ_OFTEN, (void**)&ptr); 
memcpy(texture, ptr, width * height * 4);
window->unlock();

This locks the buffer for reading, copies the data out of it, and unlocks it. Since you haven't drawn anything, there's nothing to read.

For this to do something useful, you have to render something into the texture. You can do this by creating an FBO and attaching the texture to it as the color buffer, or by using glCopyTexImage2D() to copy pixels from the framebuffer to the texture.

I was able to get your example to work by adding the following before the call to grallocBuffer->lock():

glCopyTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 0, 0, width, height, 0);
glFinish();

The glFinish() is necessary to ensure that GL has finished copying the pixels before we try to look at them.

Edit: my office-mate suggested that the GL_TEXTURE_2D needs to be GL_TEXTURE_EXTERNAL_OES. Haven't tried it yet.

Edit: see also this question.

Answer 2

GraphicBuffer is a part of Android Source Code, not in NDK. see this for your reference: https://github.com/fuyufjh/GraphicBuffer