Can\'t understand how to get (x\', y\') of original (x, y) in image, for Barrel/Pincushion distortion.
According to Wikipedia, there can also be an r to the power 4 term too. The signs of the two constants (for the r to the 2 and r to the 4 terms) can be opposite giving handlebar distortion where the centre of the image has barrel distortion and the edge has pincushion distortion giving straight lines the appearance of a handlebar moustache.
Section 2 of this paper explains the transformation. Basically:

Here I made an example in Mathematica:
An approximation of the polynomial radial distortion model you can find in Fitzgibbon, 2001 is
where rd and ru are the distances from the center of distortion. This is also used to filter the distortion out of a wide-angle camera image for computer vision and image processing purposes.
You can find a more detailed explanation of the principle and the shader code to implement the undistortion filtering (and also the forward transformation) here: http://marcodiiga.github.io/radial-lens-undistortion-filtering
I'm also posting the papers you should take a look at if you want to know the mathematical details for the method I posted
simple barrel\pincushion distortion in opencv c++
IplImage* barrel_pincusion_dist(IplImage* img, double Cx,double Cy,double kx,double ky)
{
IplImage* mapx = cvCreateImage( cvGetSize(img), IPL_DEPTH_32F, 1 );
IplImage* mapy = cvCreateImage( cvGetSize(img), IPL_DEPTH_32F, 1 );
int w= img->width;
int h= img->height;
float* pbuf = (float*)mapx->imageData;
for (int y = 0; y < h; y++)
{
for (int x = 0; x < w; x++)
{
float u= Cx+(x-Cx)*(1+kx*((x-Cx)*(x-Cx)+(y-Cy)*(y-Cy)));
*pbuf = u;
++pbuf;
}
}
pbuf = (float*)mapy->imageData;
for (int y = 0;y < h; y++)
{
for (int x = 0; x < w; x++)
{
*pbuf = Cy+(y-Cy)*(1+ky*((x-Cx)*(x-Cx)+(y-Cy)*(y-Cy)));
++pbuf;
}
}
/*float* pbuf = (float*)mapx->imageData;
for (int y = 0; y < h; y++)
{
int ty= y-Cy;
for (int x = 0; x < w; x++)
{
int tx= x-Cx;
int rt= tx*tx+ty*ty;
*pbuf = (float)(tx*(1+kx*rt)+Cx);
++pbuf;
}
}
pbuf = (float*)mapy->imageData;
for (int y = 0;y < h; y++)
{
int ty= y-Cy;
for (int x = 0; x < w; x++)
{
int tx= x-Cx;
int rt= tx*tx+ty*ty;
*pbuf = (float)(ty*(1+ky*rt)+Cy);
++pbuf;
}
}*/
IplImage* temp = cvCloneImage(img);
cvRemap( temp, img, mapx, mapy );
cvReleaseImage(&temp);
cvReleaseImage(&mapx);
cvReleaseImage(&mapy);
return img;
}
more complicated form http://opencv.willowgarage.com/documentation/camera_calibration_and_3d_reconstruction.html