ARKit 3.0 – Replicating robot character in Motion Capture RealityKit

问题

I'm trying to make a 3d model like robot provided by Apple in Motion Capture example (shown at WWDC 2019) which can mimic me in motion capture ARKit 3.0 by replacing robot character given by Apple.

Desired Solution:

• Is there any special software which Apple used to create robot.usdz file? If yes, then please provide details for it?

• How can we convert formats like .glb/.gltf/.obj/.dae file to .usdz using Apple’s Python based tool without affecting it’s scene graph?

• How can we edit the scene graph of a .usdz file in Xcode and successfully save the changes in a .usdz file?

回答1:

I haven't tested it yet.

Try the following solution to prepare a skeletal mesh for using in ARKit's MoCap:

• Create a character in Autodesk Maya 2020
• Create a skeleton, then orient axis for each joint
• Add skeleton to a mesh using Skin–Bind Skin
• Select skeleton and mesh in Outliner and use File–Game Exporter
• Choose file type: Binary and version: FBX 2020 and press Save

Then download usdz Tools for Xcode 11 and FBX Python SDK.

Here's my post where you'll find instructions how to create a .zshrc file in macOS Catalina.

Add to that .zshrc file the following line (and save it):

export PYTHONPATH="/Applications/Autodesk/FBXPythonSDK/2020.0.1/lib/Python27_ub:$PYTHONPATH"

Now you can convert .fbx file to .usdz format using our favourite command:

usdzconvert ~/Desktop/character.fbx

And test a model in mocap:

import RealityKit
import ARKit

class ViewController: UIViewController, ARSessionDelegate {

    @IBOutlet var arView: ARView!

    var character: Entity?
    let characterAnchor = AnchorEntity()

    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)

        arView.session.delegate = self

        guard ARBodyTrackingConfiguration.isSupported
        else { fatalError("MoCap is available on A12 & later") }

        let config = ARBodyTrackingConfiguration()
        arView.session.run(config)
        arView.scene.addAnchor(characterAnchor)

        character = try? Entity.load(named: "character")
    }

    func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {

        for anchor in anchors {

            guard let bodyAnchor = anchor as? ARBodyAnchor
            else { continue }

            let bodyPosition = simd_make_float3(bodyAnchor.transform.columns.3)
            characterAnchor.position = bodyPosition
            characterAnchor.orientation = Transform(matrix: bodyAnchor.transform).rotation

            if let character = character, character.parent == nil {

                characterAnchor.addChild(character)
                characterAnchor.scale = [0.02, 0.02, 0.02]
            }
        }
    }
}

P.S. Frankly saying, I have not tested ARBodyTrackingConfiguration with a custom rigged model, 'cause I have no supported device at the moment. Please, test it and write about mocap experience.

来源：https://stackoverflow.com/questions/59563842/arkit-3-0-replicating-robot-character-in-motion-capture-realitykit