(Project Tango) Rotation and translation of point clouds with area learning

Question

I have a java application that, when I press a button, records point clouds xyz coordinates together with the right pose.

What I want is to pick an object, record a pointCloud in the front and one in the back, then merge the 2 clouds.

Obviously to get a reasonable result I need to translate and rotate one or both the clouds I recorded. But I'm new to Tango Project and there are some things I should be missing.

I have read about this in this post.

There, @Jason Guo talks about those matrix:

start_service_T_device, imu_T_device , imu_T_depth

• How could I get them?
• Should i use getMatrixTransformAtTime()?

The first matrix is from start of service to device, but I'm using area learning, so my BaseFrame is TangoPoseData.COORDINATE_FRAME_AREA_DESCRIPTION.

• Is it possible to use same strategy also in my case?
• Just replacing start_service_T_device with something like area_description_T_device?

Side question

I want to extend this approach for the 3D reconstruction of objects.
I want to get several pointClouds of different view of the same object, rotate and translate them wrf some fixed axes. Then i'll assume that 2 points (x,y,z) and (x',y',z') are the same point if x~=x' && y~=y' && z~=z'.
This way i'll should be able to get a point cloud of the entire object, am I right?

• Is this approach suitable?
• Is there better alternatives?

Answer 1

The original post is a little bit out of date. Previously, we don't have getMatrixTransformAtTime(). So you have to use Tango.getPoseAtTime to query each of the transformation, and then chain them up using matrix.

But now, with getMatrixTransformAtTime, you can directly query area_description_T_depth, even in opengl frame. In order to transform a point cloud to the ADF frame in opengl, you can use following code (pseudo code):

TangoSupport.TangoMatrixTransformData transform =
  TangoSupport.getMatrixTransformAtTime(pointCloud.timestamp,
          TangoPoseData.COORDINATE_FRAME_START_OF_SERVICE,
          TangoPoseData.COORDINATE_FRAME_CAMERA_DEPTH,
          TangoSupport.TANGO_SUPPORT_ENGINE_OPENGL,
          TangoSupport.TANGO_SUPPORT_ENGINE_TANGO);

// Convert it into the matrix format you use in render.
// This is a pure data structure conversion, transform is
// in opengl world frame already.
Matrix4x4 model_matrix = ConvertMatrix(transform);

foreach (Point p in pointCloud) {
  p = model_matrix * p;
}

// Now p is in opengl world frame.

But note that, you have to have a valid area description frame to query the pose based on area description, that is after relocalized with an ADF or in learning mode.