Converting a 2D image point to a 3D world point

Question

I know that in the general case, making this conversion is impossible since depth information is lost going from 3d to 2d.

However, I have a fixed camera and I know

Answer 1

If you have Z=0 for you points in world coordinates (which should be true for planar calibration pattern), instead of inversing rotation transformation, you can calculate homography for your image from camera and calibration pattern.

When you have homography you can select point on image and then get its location in world coordinates using inverse homography. This is true as long as the point in world coordinates is on the same plane as the points used for calculating this homography (in this case Z=0)

This approach to this problem was also discussed below this question on SO: Transforming 2D image coordinates to 3D world coordinates with z = 0