stereoCalibrate() changes focal lengths even when it was not supposed to

Question

I noticed that opencv stereoCalibrate() changes the focal lengths in camera matrices even though I've set appropriate flag (ie CV_CALIB_FIX_FOCAL_LENGTH). I'm using two identical cameras with the same focal length set mechanically on lens and furthermore I know the sensor size so I can compute intrinsic camera matrix manually what actually I do.

Here you have some output form the stereo calibration program - camera matrices before and after stereoCalibrate().

    std::cout << "Before calibration: " << std::endl;
    std::cout << "C1: " << _cameraMatrixA << std::endl;
    std::cout << "C2: " << _cameraMatrixB << std::endl;

    double error = cv::stereoCalibrate(objectPoints, imagePointsA, imagePointsB, _cameraMatrixA, _distCoeffsA, _cameraMatrixB, _distCoeffsB, _imageSize,
        R, T, E, F, 
        cv::TermCriteria((cv::TermCriteria::COUNT + cv::TermCriteria::EPS), 30, 9.999999999999e-7), CV_CALIB_FIX_FOCAL_LENGTH | CV_CALIB_FIX_PRINCIPAL_POINT);

    std::cout << "After calibration: " << std::endl;
    std::cout << "C1: " << _cameraMatrixA << std::endl;
    std::cout << "C2: " << _cameraMatrixB << std::endl;

Before calibration:

C1: [6203.076923076923, 0, 1280; 0, 6203.076923076923, 960; 0, 0, 1]

C2: [6203.076923076923, 0, 1280; 0, 6203.076923076923, 960; 0, 0, 1]

After calibration:

C1: [6311.77650416514, 0, 1279.5; 0, 6331.34531760757, 959.5; 0, 0, 1]

C2: [6152.655897294907, 0, 1279.5; 0, 6206.591406832492, 959.5; 0, 0, 1]

I think this is weird opencv behavior. Anyone faced similar problem? I know it is easy to solve, I can just set focal lengths to camera matrices after stereo calibration.

Answer 1

In order to do what you want, you have to call stereoCalibrate with flags:

CV_CALIB_USE_INTRINSIC_GUESS | CV_CALIB_FIX_FOCAL_LENGTH | CV_CALIB_FIX_PRINCIPAL_POINT

If you do not use the CV_CALIB_USE_INTRINSIC_GUESS flag, stereoCalibrate will first initialize the camera matrices and distortion coefficients itself and then fix part of them in the subsequent optimization. This is stated in the documentation, although rather unclearly and without mentionning that critical flag:

Besides the stereo-related information, the function can also perform a full calibration of each of two cameras. However, due to the high dimensionality of the parameter space and noise in the input data, the function can diverge from the correct solution. If the intrinsic parameters can be estimated with high accuracy for each of the cameras individually (for example, using calibrateCamera() ), you are recommended to do so [...].

Using CV_CALIB_USE_INTRINSIC_GUESS in addition to any of the CV_CALIB_FIX_* flags tells the function to use what you are passing as input, otherwise, this input is simply ignored and overwritten.

Answer 2

the CV_CALIB_FIX_FOCAL_LENGTH flag causes the optimization routine to just use the Fx and Fy that were passed in the intrinsic matrix.