Two 3D point cloud transformation matrix

Question

I'm trying to guess wich is the rigid transformation matrix between two 3D points clouds. The two points clouds are those ones:

• keypoints from the kinect (kinect_keypoints).
• keypoints from a 3D object (box) (object_keypoints).

I have tried two options:

[1]. Implementation of the algorithm to find rigid transformation.

**1.Calculate the centroid of each point cloud.**

**2.Center the points according to the centroid.**

**3. Calculate the covariance matrix**
cvSVD( &_H, _W, _U, _V,  CV_SVD_U_T );
cvMatMul( _V,_U, &_R );
**4. Calculate the rotartion matrix using the SVD descomposition of the covariance matrix**

float _Tsrc[16] = { 1.f,0.f,0.f,0.f,
    0.f,1.f,0.f,0.f,
    0.f,0.f,1.f,0.f,
    -_gc_src.x,-_gc_src.y,-_gc_src.z,1.f };  // 1: src points to the origin
float _S[16] = { _scale,0.f,0.f,0.f,
    0.f,_scale,0.f,0.f,
    0.f,0.f,_scale,0.f,
    0.f,0.f,0.f,1.f };  // 2: scale the src points
float _R_src_to_dst[16] = { _Rdata[0],_Rdata[3],_Rdata[6],0.f, 
    _Rdata[1],_Rdata[4],_Rdata[7],0.f,
    _Rdata[2],_Rdata[5],_Rdata[8],0.f,
    0.f,0.f,0.f,1.f }; // 3: rotate the scr points
float _Tdst[16] = { 1.f,0.f,0.f,0.f, 
    0.f,1.f,0.f,0.f, 
    0.f,0.f,1.f,0.f, 
    _gc_dst.x,_gc_dst.y,_gc_dst.z,1.f }; // 4: from scr to dst

// _Tdst * _R_src_to_dst * _S * _Tsrc
mul_transform_mat( _S, _Tsrc, Rt );
mul_transform_mat( _R_src_to_dst, Rt, Rt );
mul_transform_mat( _Tdst, Rt, Rt );       

[2]. Use estimateAffine3D from opencv.

        float _poseTrans[12];
        std::vector<cv::Point3f> first, second;             
        cv::Mat aff(3,4,CV_64F, _poseTrans);
        std::vector<cv::Point3f> first, second; (first-->kineckt_keypoints and second-->object_keypoints)
        cv::estimateAffine3D( first, second, aff, inliers );

        float _poseTrans2[16];

        for (int i=0; i<12; ++i)
        {
            _poseTrans2[i] = _poseTrans[i];
        }

        _poseTrans2[12] = 0.f;
        _poseTrans2[13] = 0.f;
        _poseTrans2[14] = 0.f;
        _poseTrans2[15] = 1.f;

The problem in the first one is that the transformation it is not correct and in the second one, if a multiply the kinect point cloud with the resultant matrix, some values are infinite.

Is there any solution from any of these options? Or an alternative one, apart from the PCL?

Thank you in advance.

Answer 1

EDIT: This is an old post, but an answer might be useful to someone ...

Your first approach can work in very specific cases (ellipsoid point clouds or very elongated shapes), but is not appropriate for point clouds acquired by the kinect. And about your second approach, I am not familiar with OpenCV function estimateAffine3D but I suspect it assumes the two input point clouds correspond to the same physical points, which is not the case if you used a kinect point cloud (which contain noisy measurements) and points from an ideal 3D model (which are perfect).

You mentioned that you are aware of the Point Cloud Library (PCL) and do not want to use it. If possible, I think you might want to reconsider this, because PCL is much more appropriate than OpenCV for what you want to do (check the tutorial list, one of them covers exactly what you want to do: Aligning object templates to a point cloud).

However, here are some alternative solutions to your problem:

1. If your two point clouds correspond exactly to the same physical points, your second approach should work, but you can also check out Absolute Orientation (e.g. Matlab implementation)

2. If your two point clouds do not correspond to the same physical points, you actually want to register (or align) them and you can use either:

   • one of the many variants of the Iterative Closest Point (ICP) algorithm, if you know approximately the position of your object. Wikipedia Entry

   • 3D feature points such as 3D SIFT, 3D SURF or NARF feature points, if you have no clue about your object's position.

Again, all these approaches are already implemented in PCL.