GPGPU programming with OpenGL ES 2.0

Question

I am trying to do some image processing on the GPU, e.g. median, blur, brightness, etc. The general idea is to do something like this framework from GPU Gems 1.

I am able to write the GLSL fragment shader for processing the pixels as I've been trying out different things in an effect designer app.

I am not sure however how I should do the other part of the task. That is, I'd like to be working on the image in image coords and then outputting the result to a texture. I am aware of the gl_FragCoords variable.

As far as I understand it it goes like that: I need to set up a view (an orthographic one maybe?) and a quad in such a way so that the pixel shader would be applied once to each pixel in the image and so that it would be rendering to a texture or something. But how can I achieve that considering there's depth that may make things somewhat awkward to me...

I'd be very grateful if anyone could help me with this rather simple task as I am really frustrated with myself.

UPDATE:

It seems I'll have to use an FBO, getting one like this: glBindFramebuffer(...)

Answer 1

Basically, you need 4 vertex positions (as vec2) of a quad (with corners (-1,-1) and (1,1)) passed as a vertex attribute.

You don't really need a projection, because the shader will not need any.

Create an FBO, bind it and attach the target surface. Don't forget to check the completeness status. Bind the shader, set up input textures and draw the quad.

Your vertex shader may look like this:

#version 130
in vec2 at_pos;
out vec2 tc;
void main() {
    tc = (at_pos+vec2(1.0))*0.5;        //texture coordinates
    gl_Position = vec4(at_pos,0.0,1.0); //no projection needed
}

And a fragment shader:

#version 130
in vec2 tc;
uniform sampler2D unit_in;
void main() {
    vec4 v = texture2D(unit_in,tc);
    gl_FragColor = do_something();
}

Answer 2

Use this tutorial, it's targeted at OpenGL 2.0, but most features are available in ES 2.0, the only thing i have doubts is floating point textures.

http://www.mathematik.uni-dortmund.de/~goeddeke/gpgpu/tutorial.html

Answer 3

If you want an example, I created this project for iOS devices for processing frames of video grabbed from the camera using OpenGL ES 2.0 shaders. I explain more about it in my writeup here.

Basically, I pull in the BGRA data for a frame and create a texture from that. I then use two triangles to generate a rectangle and map the texture on that. A shader is used to directly display the image onscreen, perform some effect on the image and display it, or perform some effect on the image while in an offscreen FBO. In the last case, I can then use glReadPixels() to pull in the image for some CPU-based processing, but ideally I want to fix this so that the processed image is just passed on as a texture to the next set of shaders.

Answer 4

You should also check out ogles_gpgpu, which even supports Android systems. An overview about this topic is given in this publication: Parallel Computing for Digital Signal Processing on Mobile Device GPUs.

You can do more advanced GPGPU things with OpenGL ES 3.0 now. Check out this post for example. Apple now also has the "Metal API" which allows even more GPU compute operations. Both, OpenGL ES 3.x and Metal are only supported by newer devices with A7 chip.