Using GLM's UnProject

Question

I\'m not sure how to use the Unproject method provided by GLM.

Specifically, in what format is the viewport passed in? And why doesn\'t the function require a view m

Answer 1

The viewport is passed in as four floats: the x and y window coordinates of the viewport, followed by its width and height. That's the same order as used e.g. by glGetFloatv(GL_VIEWPORT, ...). So in most cases, the first two values should be 0.

As KillianDS already pointed out, the modelargument in fact is a modelview matrix, see the example use of unProject() in gtx_simd_mat4.cpp, function test_compute_gtx():

    glm::mat4 E = glm::translate(D, glm::vec3(1.4f, 1.2f, 1.1f));
    glm::mat4 F = glm::perspective(i, 1.5f, 0.1f, 1000.f);
    glm::mat4 G = glm::inverse(F * E);
    glm::vec3 H = glm::unProject(glm::vec3(i), G, F, E[3]);

As you can see, the matrix passed as the second argument basically is the product of a translation and a perspective transformation.