# MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild Source: [MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild]() Collection index: [llms.txt]() Data updated: 2025-10-25T20:28:18.594Z Authors: Gr\\'egory Rogez, Cordelia Schmid Published: 2016-07-07T15:30:05.000Z ## Abstract This paper addresses the problem of 3D human pose estimation in the wild. A significant challenge is the lack of training data, i.e., 2D images of humans annotated with 3D poses. Such data is necessary to train state-of-the-art CNN architectures. Here, we propose a solution to generate a large set of photorealistic synthetic images of humans with 3D pose annotations. We introduce an image-based synthesis engine that artificially augments a dataset of real images with 2D human pose annotations using 3D Motion Capture (MoCap) data. Given a candidate 3D pose our algorithm selects for each joint an image whose 2D pose locally matches the projected 3D pose. The selected images are then combined to generate a new synthetic image by stitching local image patches in a kinematically constrained manner. The resulting images are used to train an end-to-end CNN for full-body 3D pose estimation. We cluster the training data into a large number of pose classes and tackle pose estimation as a K-way classification problem. Such an approach is viable only with large training sets such as ours. Our method outperforms the state of the art in terms of 3D pose estimation in controlled environments (Human3.6M) and shows promising results for in-the-wild images (LSP). This demonstrates that CNNs trained on artificial images generalize well to real images. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [3D Human Pose Estimation on Human3.6M Protocol 2 (subjects 9 and 11)](): 3D Human Pose Estimation; Human3.6M Protocol 2 (subjects 9 and 11); Avg Error (mm): 88.1. - [3D Human Pose Estimation on Human3.6M (Protocol 2)](): 3D Human Pose Estimation; Human3.6M (Protocol 2); Average MPJPE: 88.1. - [3D Human Pose Estimation on Human3.6M (Protocol #1)](): 3D Human Pose Estimation; Human3.6M (Protocol #1); MPJPE (Avg.): 88.1. - [3D Human Pose Estimation on Human3.6M (test)](): 3D Human Pose Estimation; Human3.6M (test); MPJPE (Average): not recorded. - [3D Human Pose Estimation on Human3.6M](): 3D Human Pose Estimation; Human3.6M; MPJPE: not recorded. - [3D Human Pose Estimation on Human3.6M (subjects 9 and 11)](): 3D Human Pose Estimation; Human3.6M (subjects 9 and 11); Average Error: not recorded. ## Original sources and implementation [Original paper]() [PDF]()