How do I determine what is touched in 3D space from the screen?

Question

How do I use gl.gluUnproject in my OpenGL ES 1.1 android app to determine what is selected when the user touches the screen?

My understanding is that the touch event

Answer 1

If you are doing 2D to 3D picking, you need to fiddle with matrices and vectors a bit. GlUnproject does not exist for OpenGL ES 1.1 so you have to do some math by yourself.

Ray-object intersection is a way to go then. Timmmms answer already covers some of it, but there's more. Idea is to create ray to 3D out of 2D coordinates. Inverse of view matrix and projection matrix are needed for that. Once you have ray, you can use ray-intersection test of your choice and of course you need to select closest object like at Timmmm's point 4. Bounding spheres and bounding boxes are easy to implement and internet is full of intersection test tutorials for them.

This picking tutorial is for DirectX, but you might get the idea. Ray-constructing part is most important.

Edit Android implements it's own version of gluUnproject. It can be used to create ray, by calling it for near and far plane (0 and 1) and subtracting near plane results from far plane results to get ray's direction. Ray origin is view location. More here.