How do I reverse-project 2D points into 3D?

Question

I have 4 2D points in screen-space, and I need to reverse-project them back into 3D space. I know that each of the 4 points is a corner of a 3D-rotated rigid rectangle, and

Answer 1

Yes, Monte Carlo works, but I found better solution for this issue. This code works perfectly (and uses OpenCV):

Cv2.CalibrateCamera(new List>() { points3d }, new List>() { points2d }, new Size(height, width), cameraMatrix, distCoefs, out rvecs, out tvecs, CalibrationFlags.ZeroTangentDist | CalibrationFlags.FixK1 | CalibrationFlags.FixK2 | CalibrationFlags.FixK3);

This function takes known 3d and 2d points, size of screen and returns rotation (rvecs[0]), translation (tvecs[0]) and matrix of intrinsics values of camera. It's everything you need.