# DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras

Source: [DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras](<https://www.sota2.com/research/paper/68fa6aac1e209131cea29a43>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:30:26.199Z

Authors: Yang Zheng, Ruizhi Shao, Yuxiang Zhang, Tao Yu, Zerong Zheng, Qionghai Dai, Yebin Liu

Published: 2021-05-01T14:32:13.000Z

## Abstract

We propose DeepMultiCap, a novel method for multi-person performance capture using sparse multi-view cameras. Our method can capture time varying surface details without the need of using pre-scanned template models. To tackle with the serious occlusion challenge for close interacting scenes, we combine a recently proposed pixel-aligned implicit function with parametric model for robust reconstruction of the invisible surface areas. An effective attention-aware module is designed to obtain the fine-grained geometry details from multi-view images, where high-fidelity results can be generated. In addition to the spatial attention method, for video inputs, we further propose a novel temporal fusion method to alleviate the noise and temporal inconsistencies for moving character reconstruction. For quantitative evaluation, we contribute a high quality multi-person dataset, MultiHuman, which consists of 150 static scenes with different levels of occlusions and ground truth 3D human models. Experimental results demonstrate the state-of-the-art performance of our method and the well generalization to real multiview video data, which outperforms the prior works by a large margin.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Runtime and VRAM for Multi-human 3D Reconstruction on Multi-human 3D Reconstruction dataset](<https://www.sota2.com/research/sota/runtime-and-vram-for-multi-human-3d-reconstruction-on-multi-human-3d-reconstruction-dataset.md>): Multi-human 3D reconstruction; Multi-human 3D Reconstruction dataset; Elapsed Time (s): 42.35.

- [Detailed geometry reconstruction on Hi4D](<https://www.sota2.com/research/sota/detailed-geometry-reconstruction-on-hi4d.md>): Detailed geometry reconstruction; Hi4D; IoU: 90.6.

- [3D Human Reconstruction (Occluded Regions) on MultiHuman Closely](<https://www.sota2.com/research/sota/3d-human-reconstruction-occluded-regions-on-multihuman-closely.md>): 3D Human Reconstruction (Occluded Regions); MultiHuman Closely; L2 Norm Error: 0.219.

- [3D Multi-Human Reconstruction on MultiHuman Closely interaction](<https://www.sota2.com/research/sota/3d-multi-human-reconstruction-on-multihuman-closely-interaction.md>): 3D multi-human reconstruction; MultiHuman Closely Interaction; CD: 9.6697.

- [Multi-human 3D reconstruction on MultiHuman](<https://www.sota2.com/research/sota/multi-human-3d-reconstruction-on-multihuman.md>): Multi-human 3D reconstruction; MultiHuman; CD: 13.719.

- [3D Human Reconstruction (Occluded Regions) on MultiHuman Natural](<https://www.sota2.com/research/sota/3d-human-reconstruction-occluded-regions-on-multihuman-natural.md>): 3D Human Reconstruction (Occluded Regions); MultiHuman Natural; Norm L2: 0.216.

- [Multi-human 3D reconstruction on MultiHuman occluded region](<https://www.sota2.com/research/sota/multi-human-3d-reconstruction-on-multihuman-occluded-region.md>): Multi-human 3D reconstruction; MultiHuman occluded region; Norm L2: 0.217.

- [3D Multi-Human Reconstruction on MultiHuman Natural interaction](<https://www.sota2.com/research/sota/3d-multi-human-reconstruction-on-multihuman-natural-interaction.md>): 3D multi-human reconstruction; MultiHuman Natural Interaction; CD: 17.081.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2105.00261>)

[PDF](<https://arxiv.org/pdf/2105.00261.pdf>)