1. 技术文章
  2. Handling touch events in a 3D “scene” or Screen to 3D coordinates

Handling touch events in a 3D “scene” or Screen to 3D coordinates

由 巧了我就是萌 提交于 2019-12-03 22:15:56

I remember runnning into problems with glUnProject on Android back in my college days. (That was in the early days of Android) One of my fellow students figured out that our calculations would get mangled by the 4th dimension in the result of glUnProject. If I recall correctly, this was something documented somewhere, but for some reason I haven't been able to dig that up again. I never dug into the specifics of it, but perhaps what helped us may also be of use to you. It's likely to do with the math we applied...

/**
 * Convert the 4D input into 3D space (or something like that, otherwise the gluUnproject values are incorrect)
 * @param v 4D input
 * @return 3D output
 */
private static float[] fixW(float[] v) { 
    float w = v[3];
    for(int i = 0; i < 4; i++) 
        v[i] = v[i] / w;
    return v;
}

We actually used the above method to fix up the glUnProject results and do a pick/touch/select action on spherical objects in 3D space. Below code may provide a guide on how to do this. It's little more than casting a ray and doing a ray-sphere intersection test.

A few additional notes that may make below code more easy to understand:

  • Vector3f is a custom implementation of a 3D vector based on 3 float values and implements the usual vector operations.
  • shootTarget is the spherical object in 3D space.
  • The 0 in calls like getXZBoundsInWorldspace(0) and getPosition(0) are simply an index. We implemented 3D model animations and the index determines which 'frame/pose' of the model to return. Since we ended up doing this specific hit test on a non-animated object, we always used the first frame.
  • Concepts.w and Concepts.h are simply the width and height of the screen in pixels - or perhaps differently said for a full screen app: the screen's resolution.

_

/**
 * Checks if the ray, casted from the pixel touched on-screen, hits
 * the shoot target (a sphere). 
 * @param x
 * @param y
 * @return Whether the target is hit
 */
public static boolean rayHitsTarget(float x, float y) {
    float[] bounds = Level.shootTarget.getXZBoundsInWorldspace(0);
    float radius = (bounds[1] - bounds[0]) / 2f;
    Ray ray = shootRay(x, y);
    float a = ray.direction.dot(ray.direction);  // = 1
    float b = ray.direction.mul(2).dot(ray.near.min(Level.shootTarget.getPosition(0)));
    float c = (ray.near.min(Level.shootTarget.getPosition(0))).dot(ray.near.min(Level.shootTarget.getPosition(0))) - (radius*radius);

    return (((b * b) - (4 * a * c)) >= 0 );

}

/**
 * Casts a ray from screen coordinates x and y.
 * @param x
 * @param y
 * @return Ray fired from screen coordinate (x,y)
 */
public static Ray shootRay(float x, float y){
    float[] resultNear = {0,0,0,1};
    float[] resultFar = {0,0,0,1};

    float[] modelViewMatrix = new float[16];
    Render.viewStack.getMatrix(modelViewMatrix, 0);

    float[] projectionMatrix = new float[16];
    Render.projectionStack.getMatrix(projectionMatrix, 0);

    int[] viewport = { 0, 0, Concepts.w, Concepts.h };

    float x1 = x;
    float y1 = viewport[3] - y;

    GLU.gluUnProject(x1, y1, 0.01f, modelViewMatrix, 0, projectionMatrix, 0, viewport, 0, resultNear, 0);
    GLU.gluUnProject(x1, y1, 50f, modelViewMatrix, 0, projectionMatrix, 0, viewport, 0, resultFar, 0);
    //transform the results from 4d to 3d coordinates.
    resultNear = fixW(resultNear);
    resultFar = fixW(resultFar);
    //create the vector of the ray.
    Vector3f rayDirection = new Vector3f(resultFar[0]-resultNear[0], resultFar[1]-resultNear[1], resultFar[2]-resultNear[2]);
    //normalize the ray.
    rayDirection = rayDirection.normalize();
    return new Ray(rayDirection, resultNear, resultFar);
}

/**
 * @author MH
 * Provides some accessors for a casted ray.
 */
public static class Ray {
    Vector3f direction;
    Vector3f near;
    Vector3f far;

    /**
     * Casts a new ray based on the given direction, near and far params. 
     * @param direction
     * @param near
     * @param far
     */
    public Ray(Vector3f direction, float[] near, float[] far){
        this.direction = direction;
        this.near = new Vector3f(near[0], near[1], near[2]);
        this.far = new Vector3f(far[0], far[1], far[2]);
    }
}
harism

How can I make sure I am getting the proper near and far coordinates based on the screen coordinates.

To begin with, read this answer if you haven't read information about GLU.glProject somewhere else already. GLU.glUnProject does the exact inverse of that function, which I found more easy to understand, and helps to understand the concept of mapping screen coordinates into object space. Worked for me at least.

If you want to test you're getting correct values from GLU.glUnProject it's easiest done if you start with some easy to understand projection and model-view -matrices. Here's a code snippet I was using earlier today;

  // Orthographic projection for the sake of simplicity.
  float projM[] = new float[16];
  Matrix.orthoM(projM, 0, -ratio, ratio, 1f, -1f, zNear, zFar);

  // Model-View matrix is simply a 180 degree rotation around z -axis.
  float mvM[] = new float[16];
  Matrix.setLookAtM(mvM, 0, 0f, 0f, eyeZ, 0f, 0f, 0f, 0f, 1f, 0f);
  Matrix.rotateM(mvM, 0, 180f, 0f, 0f, 1f);

  // (0, 0) is top left point of screen.
  float x = (width - (width / ratio)) / 2;
  float y = height;
  float objNear[] = new float[4];
  float objFar[] = new float[4];
  int view[] = { 0, 0 ,width, height };
  // --> objNear = { 1, 1, eyeZ - zNear  }
  GLU.gluUnProject(x, y, 0, mvM, 0, projM, 0, view, 0, objNear, 0);
  // --> objFar = { 1, 1, eyeZ - zFar }
  GLU.gluUnProject(x, y, 1, mvM, 0, projM, 0, view, 0, objFar, 0);

With more complex model-view and projection matrices it becomes rather difficult to verify you're getting coordinates in object space you're expecting. And it might be a good starting point to play around with some easy to understand matrices first. Once you're happy with the results you shouldn't have to worry about GLU.glUnProject at all.

Once I have the near and far coordinates, how do I use that to find if the line they create intersects an object on the screen.

For hit testing in 3d object space, it should be easiest to pre-compute bounding sphere or bounding box for your objects. Once you want to check if user clicked on an object, and have two screen points unProjected into object space, you have two points on a line in object space.

If you're using bounding sphere you can calculate intersection points with this line, or alternatively, only sphere center point distance. In former case should there be at least one intersection and in latter one distance should be less than sphere radius.

For bounding box this question is a good read.

标签
android
opengl-es
3d
coordinates
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!