I have a large bulk of photographs taken with a fisheye lens. As I want to do some image-processing (e.g. edge detection) on the photos I want to remove the barrel distortio
I think your circles are caused by this line:
double srcYd = centerY + (diffX * factor);
which I'm guessing should be:
double srcYd = centerY + (diffY * factor);