Android OpenGL combination of SurfaceTexture (external image) and ordinary texture

Question

I would like to mix camera preview SurfaceTexture with some overlay texture. I am using these shaders for processing:

private final String vss =          


        

Answer 1

Referring to user1924406's post (https://stackoverflow.com/a/14050597/3250829) on splitting up accessing a sampler2D texture and samplerExternalOES texture, this is what I had to do because the application that I am developing is reading from a file or streaming from a server instead of using the Camera that is on the device. Using both textures in the same shader resulted in very weird colourization artifacts (the case on a Galaxy S3) or saturation and contrast issues (the case on a Nexus 4).

As such, the only way to work around the samplerExternalOES texture bug (from what I've seen so far) is to do the two shader programs: One that writes the content contained in the samplerExternalOES texture to a FBO and the other that takes the content from the FBO and writes it directly to the surface.

One thing that you need to check is that sometimes when you write to a FBO, the texture co-ordinates flip. In my case, the V (or T or Y) co-ordinate was flipped which resulted in a mirrored image across the horizontal axis. I had to take this into account when writing the fragment shader at the second stage.

This is a war story that I wanted to share in case there are some of you that may need to read from a file or stream from a server instead of taking directly from the Camera.