julienkay/MotionBricks-inference-engine

MotionBricks for Unity Inference Engine

NVIDIA MotionBricks models for Unity's Inference Engine (formerly Sentis). MotionBricks generates motion in real time for humanoid robots and human characters: give it the character's recent poses and the poses it should reach next, and it generates the movement between them.

The Unity package tools.doji.motionbricks runs these models. It downloads them with AI Model Hub (tools.doji.model-hub), so you do not need to download them yourself.

Files

File	Model	Size
sentis/motionbricks_soma_fp32.sentis	SOMA: human motion on the SOMA skeleton (MotionBricks-SOMA-RP-v1).	757 MB
sentis/motionbricks_unitree_g1_fp32.sentis	Unitree G1 humanoid robot (MotionBricks-G1-RP-v1).	733 MB
onnx/motionbricks_soma_fp32.onnx	The SOMA model as ONNX.	758 MB
onnx/motionbricks_unitree_g1_fp32.onnx	The Unitree G1 model as ONNX.	734 MB

The .sentis files load without conversion. They were saved with Inference Engine 2.6.1 from the ONNX files; each has a .sentis.json record with the Unity and Inference Engine versions, inputs and outputs. The ONNX files are the source: import them when your Inference Engine version does not load the .sentis files, or to choose your own conversion settings. Each has an .export.json report: the upstream source and checksum, the changes, the inputs and outputs, and the operators. SHA256SUMS lists the checksums of every file.

The tag v1 marks the files a package release uses. Later uploads get a new tag.

Source and changes

Both models come from the ONNX exports in the MotionBricks software at revision 54fc563aeaaf819a753d72cc9fdc6713989cdd43 (motionbricks/assets/onnx/). They are FP32 graphs at opset 17.

• SOMA: unchanged (mmm_v6_soma_fp32.onnx).
• Unitree G1: one traced squeeze-guard If node is replaced by the equivalent Squeeze, which Inference Engine supports. The outputs are unchanged (mmm_v6_g1_fp32.onnx).

The weights are not quantized. With Inference Engine weight quantization, Float16 halves the size with the same output and GPU speed but is about 4 times slower on the CPU backend; Uint8 changes the output too much to use. Upstream also ships mixed FP16 graphs. Inference Engine converts FP16 weights to FP32 on import, so they are not faster and are not included.

Use in Unity

Add tools.doji.motionbricks to your project. Download the SOMA model in Window > AI Model Hub, or let the first MotionBricks component that starts download it. The Unitree G1 model is optional: turn it on in AI Model Hub.

using Doji.AI.MotionBricks;
using Unity.InferenceEngine;

Model model = await MotionBricksModels.LoadAsync();                       // SOMA
Model robot = await MotionBricksModels.LoadAsync(MotionBricksModels.UnitreeG1);

License

The models are licensed by NVIDIA Corporation under the NVIDIA Open Model Agreement, which permits commercial use and redistribution of modified models, subject to its terms.

This repository contains only the models. It does not contain the MotionBricks sample motions or training data. The MotionBricks software uses the Apache 2.0 license.

Citation

To cite MotionBricks, see the project page and the paper.