Android OpenGL combination of SurfaceTexture (external image) and ordinary texture

Question

I would like to mix camera preview SurfaceTexture with some overlay texture. I am using these shaders for processing:

private final String vss =          


        

Answer 1

In order to convert an external texture (non-GPU) to a regular internal one, you have to

• First create an external texture (same as creating a regular texture, but replacing all GLES20.GL_TEXTURE_2D with GLES11Ext.GL_TEXTURE_EXTERNAL_OES).
• Create an internal/regular texture to write to
• Wrap a SurfaceTexture with the external texture.
• Read the content into that external texture (if from a camera, a file, etc.)
• Override onFrameAvailable of that SurfaceTexture and do the conversion here, supplying both the external texture to read from, and the internal texture to write to.
• You might need to call getTransformMatrix for corrections in coordinates (usually flip y axis) and supply it as well. Sometimes not...

here's some example shaders:

Vertex-shdaer -

uniform mat4 transform; // might be needed, and might not
uniform mat4 modelview;
uniform mat4 projection;
attribute vec2 position;

varying vec2 vTexcoord;

void main() {
    gl_Position = projection * modelview * vec4(position.xy, 0.0, 1.0);
    // texture takes points in [0,1], while position is [-1,1];
    vec4 newpos = (gl_Position + 1.0) * 0.5;
    vTexcoord = (transform * newpos).xy ;
}

Fragment shader -

#extension GL_OES_EGL_image_external : require

precision mediump float;

uniform samplerExternalOES sTexture;
varying vec2 vTexcoord;

void main() {
    gl_FragColor = texture2D(sTexture, vTexcoord);
}