How can I get the yaw, pitch, roll of an ARAnchor in absolute terms?

问题

I've been trying to figure this out for a few days now.

Given an ARKit-based app where I track a user's face, how can I get the face's rotation in absolute terms, from its anchor?

I can get the transform of the ARAnchor, which is a simd_matrix4x4. There's a lot of info on how to get the position out of that matrix (it's the 3rd column), but nothing on the rotation!

I want to be able to control a 3D object outside of the app, by passing YAW, PITCH and ROLL.

The latest I thing I tried actually works somewhat:

let arFrame = session.currentFrame!
guard let faceAnchor = arFrame.anchors[0] as? ARFaceAnchor else { return }

let faceMatrix = SCNMatrix4.init(faceAnchor.transform)
let node = SCNNode()

node.transform = faceMatrix
let rotation = node.worldOrientation

rotation.x .y and .z have values I could use, but as I move my phone the values change. For instance, if I turn 180˚ and keep looking at the phone, the values change wildly based on the position of the phone.

I tried changing the world alignment in the ARConfiguration, but that didn't make a difference.

Am I reading the wrong parameters? This should have been a lot easier!

回答1:

I've figured it out...

Once you have the face anchor, some calculations need to happen with its transform matrix, and the camera's transform.

Like this:

let arFrame = session.currentFrame!
guard let faceAnchor = arFrame.anchors[0] as? ARFaceAnchor else { return }

let projectionMatrix = arFrame.camera.projectionMatrix(for: .portrait, viewportSize: self.sceneView.bounds.size, zNear: 0.001, zFar: 1000)
let viewMatrix = arFrame.camera.viewMatrix(for: .portrait)

let projectionViewMatrix = simd_mul(projectionMatrix, viewMatrix)
let modelMatrix = faceAnchor.transform
let mvpMatrix = simd_mul(projectionViewMatrix, modelMatrix)

// This allows me to just get a .x .y .z rotation from the matrix, without having to do crazy calculations
let newFaceMatrix = SCNMatrix4.init(mvpMatrix)
let faceNode = SCNNode()
faceNode.transform = newFaceMatrix
let rotation = vector_float3(faceNode.worldOrientation.x, faceNode.worldOrientation.y, faceNode.worldOrientation.z)

rotation.x .y and .z will return the face's pitch, yaw, roll (respectively)

I'm adding a small multiplier and inverting 2 of the axis, so it ends up like this:

yaw = -rotation.y*3
pitch = -rotation.x*3
roll = rotation.z*1.5

Phew!

回答2:

I understand that you are using front camera and ARFaceTrackingConfiguration, which is not supposed to give you absolute values. I would try to configure second ARSession for back camera with ARWorldTrackingConfiguration which does provide absolute values. The final solution will probably require values from both ARSession's. I haven't tested this hypothesis yet but it seems to be the only way.

UPDATE quote from ARWorldTrackingConfiguration -

The ARWorldTrackingConfiguration class tracks the device's movement with six degrees of freedom (6DOF): specifically, the three rotation axes (roll, pitch, and yaw), and three translation axes (movement in x, y, and z). This kind of tracking can create immersive AR experiences: A virtual object can appear to stay in the same place relative to the real world, even as the user tilts the device to look above or below the object, or moves the device around to see the object's sides and back.

Apparently, other tracking configurations do not have this ability.

来源：https://stackoverflow.com/questions/53352476/how-can-i-get-the-yaw-pitch-roll-of-an-aranchor-in-absolute-terms

标签

swift

arkit