glm::lookAt vertical camera flips when z <= 0

Question

I'm working on a FPS style camera to fly around my 3D scene using OpenGL. I'm using GLM for the mathmetics and I calculate a direction vector with a glm::rotate on the x-axis and the y-axis using mouse movements. I have a static up vector since I'm fine with strafing on the horizontal axis and don't really need any rolls.

However, when I move forward towards the negative z direction and I eventually reach the center point of the scene (z=0) the vertical camera flips (y direction). Moving the mouse downwards will now result in an upward motion. The direction vectors are calculated like they should so I'm guessing it has something to do with how glm::lookAt calculates its view Matrix?

Here is the relevant code:

// Player movement
glm::vec3 position(0.0f, 0.0f, 5.0f);
glm::vec3 direction(0.0f, 0.0f, -1.0f);
glm::vec3 up(0.0f, 1.0f, 0.0f);
float speed = 0.05f;
bool warped = false;
...

void render()
{
    ...
    view = glm::lookAt(position, position + direction, up); 
    glUniformMatrix4fv(glGetUniformLocation(basicShader.shaderProgram, "view"), 1, GL_FALSE, glm::value_ptr(view));
    ...    
}


void handleKeyboard()
{
    float speed = 0.05f;
    if(keys['w'])
        position += speed * direction;
    if(keys['s'])
        position -= speed * direction;
    if(keys['a'])
        position -= speed * glm::cross(direction, up);
    if(keys['d'])
        position += speed * glm::cross(direction, up); 
    if(keys['q'])
        exit(1);
}

// Set with following callbacks:
// glutMotionFunc(mouse);
// glutPassiveMotionFunc(mouse);
void mouse(int x, int y)
{  
    if(!warped)
    {
        float mouseSensitivity = 15.0f;

        float horizontal = (width / 2) - x;
        float vertical = (height / 2) - y;

        horizontal /= mouseSensitivity;
        vertical /= mouseSensitivity;

        direction = glm::rotate(direction, horizontal, glm::vec3(0.0f, 1.0f, 0.0f));
        direction = glm::rotate(direction, vertical, glm::vec3(1.0f, 0.0f, 0.0f));

        warped = true;
        glutWarpPointer((width / 2), (height / 2));
    }
    else
        warped = false;
}

Answer 1

I'm not entirely sure what the problem is, but when I use LookAt for an FPS cam I also include a forward unit vector.

E.g (copied relevant segments from some code I have) - using quaternions (orientation), but I'm sure matrices will have the same result)

static glm::vec3 defaultUpVector() { return glm::vec3(0, 1, 0); }
static glm::vec3 defaultForwardVector() { return glm::vec3(0, 0, -1); }

pUpVector = defaultUpVector() * orientation;
pLookAt = position + (defaultForwardVector() * orientation);
pView = glm::lookAt(position, pLookAt, pUpVector);

upvector and lookat are both vec3, and view is mat4

Hope this helps.

EDIT: I notice you are using a direction, so I would probably look at where you use rotate.

This will set the quat for an FPS cam with no roll

pLocalOrientation = 
    glm::angleAxis(pLocalEularAngles.x, glm::vec3(1, 0, 0)) *
    glm::angleAxis(pLocalEularAngles.y, glm::vec3(0, 1, 0));

pLocalOrientation == orientation (for the example)

If you decide to use quats.