# Reconstructing Animatable Categories from Videos

Source: [Reconstructing Animatable Categories from Videos](<https://www.sota2.com/research/paper/68fa6f451e209131cea44ab3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:28:45.407Z

Authors: Gengshan Yang, Chaoyang Wang, N Dinesh Reddy, Deva Ramanan

Published: 2023-05-10T17:56:21.000Z

## Abstract

Building animatable 3D models is challenging due to the need for 3D scans, laborious registration, and manual rigging, which are difficult to scale to arbitrary categories. Recently, differentiable rendering provides a pathway to obtain high-quality 3D models from monocular videos, but these are limited to rigid categories or single instances. We present RAC that builds category 3D models from monocular videos while disentangling variations over instances and motion over time. Three key ideas are introduced to solve this problem: (1) specializing a skeleton to instances via optimization, (2) a method for latent space regularization that encourages shared structure across a category while maintaining instance details, and (3) using 3D background models to disentangle objects from the background. We show that 3D models of humans, cats, and dogs can be learned from 50-100 internet videos.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Surface Reconstruction on AMA samba](<https://www.sota2.com/research/sota/surface-reconstruction-on-ama-samba.md>): Surface Reconstruction; AMA samba; CD: 6.

- [Surface Reconstruction on AMA bouncing](<https://www.sota2.com/research/sota/surface-reconstruction-on-ama-bouncing.md>): Surface Reconstruction; AMA bouncing; CD: 6.9.

- [3D Human Reconstruction on MonoPerfCap Pablo sequence](<https://www.sota2.com/research/sota/3d-human-reconstruction-on-monoperfcap-pablo-sequence.md>): 3D human reconstruction; MonoPerfCap Pablo sequence; 3D Chamfer Distance (cm): 18.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2305.06351>)

[PDF](<https://arxiv.org/pdf/2305.06351.pdf>)

[Implementation](<https://gengshan-y.github.io/rac-www/>)