2D Coordinate to 3D world coordinate

Question

I want to convert 2D Image coordinates to 3D world coordinates. I am using the ZED camera which is a stereo camera and the sdk shipped with it, provides the disparity map. H

Answer 1

As you already know the depth D in meters for each pixel, you don't need the baseline B between cameras (which you would need to compute the depth from the disparity value). In fact, D is already your searched Z coordinate.

The general formula for the pinhole camera model (assuming there is no distortion) is:

u = fx * (X / Z) + cx
v = fy * (Y / Z) + cy

So it is then straightforward to compute the 3D-coordinates:

X = Z / fx * (u - cx)
Y = Z / fy * (v - cy)
[Z = D]

Note that this only is correct if you are working with a rectified image (or an image with low distortion).