OpenCV cvRemap Cropping Image

Question

So I am very new to OpenCV (2.1), so please keep that in mind.

So I managed to calibrate my cheap web camera that I am using (with a wide angle attachment), using the checkerboard calibration method to produce the intrinsic and distortion coefficients.

I then have no trouble feeding these values back in and producing image maps, which I then apply to a video feed to correct the incoming images.

I run into an issue however. I know when it is warping/correcting the image, it creates several skewed sections, and then formats the image to crop out any black areas. My question then is can I view the complete warped image, including some regions that have black areas? Below is an example of the black regions with skewed sections I was trying to convey if my terminology was off:

An image better conveying the regions I am talking about can be found here! This image was discovered in this post.

Currently: The cvRemap() returns basically the yellow box in the image linked above, but I want to see the whole image as there is relevant data I am looking to get out of it.

What I've tried: Applying a scale conversion to the image map to fit the complete image (including stretched parts) into frame

        CvMat *intrinsic = (CvMat*)cvLoad( "Intrinsics.xml" );
        CvMat *distortion = (CvMat*)cvLoad( "Distortion.xml" );

        cvInitUndistortMap( intrinsic, distortion, mapx, mapy );

        cvConvertScale(mapx, mapx, 1.25, -shift_x);   // Some sort of scale conversion
        cvConvertScale(mapy, mapy, 1.25, -shift_y);   // applied to the image map

        cvRemap(distorted,undistorted,mapx,mapy);

The cvConvertScale, when I think I have aligned the x/y shift correctly (guess/checking), is somehow distorting the image map making the correction useless. There might be some math involved here I am not correctly following/understanding.

Does anyone have any other suggestions to solve this problem, or what I might be doing wrong? I've also tried trying to write my own code to fix distortion issues, but lets just say OpenCV knows already how to do it well.

Answer 1

From memory, you need to use InitUndistortRectifyMap(cameraMatrix,distCoeffs,R,newCameraMatrix,map1,map2), of which InitUndistortMap is a simplified version.

cvInitUndistortMap( intrinsic, distort, map1, map2 )

is equivalent to:

cvInitUndistortRectifyMap( intrinsic, distort, Identity matrix, intrinsic, 
                           map1, map2 )

The new parameters are R and newCameraMatrix. R species an additional transformation (e.g. rotation) to perform (just set it to the identity matrix).

The parameter of interest to you is newCameraMatrix. In InitUndistortMap this is the same as the original camera matrix, but you can use it to get that scaling effect you're talking about.

You get the new camera matrix with GetOptimalNewCameraMatrix(cameraMat, distCoeffs, imageSize, alpha,...). You basically feed in intrinsic, distort, your original image size, and a parameter alpha (along with containers to hold the result matrix, see documentation). The parameter alpha will achieve what you want.

I quote from the documentation:

The function computes the optimal new camera matrix based on the free scaling parameter. By varying this parameter the user may retrieve only sensible pixels alpha=0, keep all the original image pixels if there is valuable information in the corners alpha=1, or get something in between. When alpha>0, the undistortion result will likely have some black pixels corresponding to “virtual” pixels outside of the captured distorted image. The original camera matrix, distortion coefficients, the computed new camera matrix and the newImageSize should be passed to InitUndistortRectifyMap to produce the maps for Remap.

So for the extreme example with all the black bits showing you want alpha=1.

In summary:

• call cvGetOptimalNewCameraMatrix with alpha=1 to obtain newCameraMatrix.
• use cvInitUndistortRectifymap with R being identity matrix and newCameraMatrix set to the one you just calculated
• feed the new maps into cvRemap.