World-Grounded Human Motion Recovery via Gravity-View Coordinates
About
We present a novel method for recovering world-grounded human motion from monocular video. The main challenge lies in the ambiguity of defining the world coordinate system, which varies between sequences. Previous approaches attempt to alleviate this issue by predicting relative motion in an autoregressive manner, but are prone to accumulating errors. Instead, we propose estimating human poses in a novel Gravity-View (GV) coordinate system, which is defined by the world gravity and the camera view direction. The proposed GV system is naturally gravity-aligned and uniquely defined for each video frame, largely reducing the ambiguity of learning image-pose mapping. The estimated poses can be transformed back to the world coordinate system using camera rotations, forming a global motion sequence. Additionally, the per-frame estimation avoids error accumulation in the autoregressive methods. Experiments on in-the-wild benchmarks demonstrate that our method recovers more realistic motion in both the camera space and world-grounded settings, outperforming state-of-the-art methods in both accuracy and speed. The code is available at https://zju3dv.github.io/gvhmr/.
Key insights
How does GVHMR recover world-grounded motion?
It estimates per-frame human poses in a Gravity-View coordinate system defined by gravity and camera view direction, then transforms them into a consistent world frame using camera rotations, avoiding autoregressive error accumulation.
What distinguishes GVHMR from prior world-motion methods?
GVHMR predicts motion in parallel instead of accumulating autoregressive errors. On EMDB-2, it reports 109.1 mm world-aligned mean joint-position error over 100-frame segments, compared with WHAM's 131.1 mm; lower error is better.
Where can world-grounded motion be useful?
The authors motivate motion datasets for text-to-motion generation and humanoid imitation. Their core network processes 1,430 frames in 280 ms on an RTX 4090; this timing excludes preprocessing and does not represent an entire deployed capture pipeline.
What limitations do the authors acknowledge?
The method predicts SMPLX rather than SMPL parameters, which may introduce errors; PA-MPJPE on 3DPW trails WHAM by 0.3mm. Evaluation is limited to three in-the-wild benchmarks (3DPW, RICH, EMDB) with sequence lengths up to 120 frames during training.