I am using OpenCV to calibrate images taken using cameras with fish-eye lenses.
The functions I am using are:
findChessboardCorners(...);
I stacked the same problem. And if FOV of your camera ~ 180 degrees, I think you will not be able to undistort 100% of initial image surface. More detailed explanation I placed here