Visually correct rotation with OpenGl and touch screen capabilities

Question

I have been trying to do Cube of Rubik for android. I have one question about rotations. I want to rotate a figure visually correct. It means if user touch screen and after move his finger to right a figure rotate to right from side of observation point. But when I make some rotations the figure start move in not correct direction. I understand that it depends on that axis are change their situation. But I tried to use inverse model matrix to get necessary coordinates, but I haven't already result. Could anybody give me example or link of visually correct rotation of 3D figure with help of mouse or touch screen?

   //get vector 3D of touch
   Vector3f touchVector = getRubikSystemCoordinates(mTouchX,mTouchY,square.rubikRotationMatrix);
   //Get vector 3D of move     
   Vector3f moveVector = getRubikSystemCoordinates(mMoveX,mMoveY,square.rubikRotationMatrix);
        //get direction of motion
        float direction = touchVector.substractFrom(moveVector); 
        //get axis for rotation
        Vector3f axis = touchVector.vectorProductTo(moveVector);
        //normalize axis
        axis.normalize();
        //get angle of rotation
        float angle = direction.length;
        //make identity Quad
        Quaternion quad = new Quaternion();
        //make rotation quad
        quad.makeRotationKvaternion(angle,axis);
        //from quad recieve matrix
        Matrix4f matrix = quad.toMatrix();
        //multiply to current modelview matrix
        gl.glMultMatrixf(matrix.returnArray(),0);
        //save rotation matrix
        square.rotationMatrix = square.rotationMatrix.multiply(matrix);
        //save modelView matrix
        square.saveModelView(square.initMatrix.returnArray()); 


      // touch coords to current modelView coords
      private Vector3f getRubikSystemCoordinates(float x, float y, Matrix4f matrix){
       // touch coords to normal coords of screen 
       Vector2f normalCoords = (new Vector2f(x,y)).toNormalScreenCoordinates(Settings.viewPort[2],Settings.viewPort[3]);
        // to sphere coords in 3D
        Vector3f sphereVector = new Vector3f(normalCoords.x,normalCoords.y, FloatMath.sqrt(2-normalCoords.x*normalCoords.x-normalCoords.y*normalCoords.y));
        //Get inverse matrix from ModelView Matrix
        Matrix4f m = matrix.inverseMatrix();
        //Get vector for current modelView 3D coords
        Vector3f vector = m.multiplyToVector(vector);
        // make normalize vector
        vector.normalize();
        return vector;
        }

Answer 1

What you're looking for is named arcball rotation. You'll find plenty of resources in java around the internet on this.

Answer 2

You are probably storing your rotation as three angles. Use matrix instead. Create a separate transformation matrix just for this rotation. Every time user rotates the object apply the rotation to this matrix. This way the movement will always be relative to current orientation.