# EgoAERO: Learning Dexterous Manipulation from a Single Egocentric Video without Object Assets

Source: [EgoAERO: Learning Dexterous Manipulation from a Single Egocentric Video without Object Assets](<https://www.sota2.com/research/paper/6a279e3d90966ade919bf83e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-10T05:00:46.690Z

Authors: Yichen Niu, Haoran Lv, Xinrui Zhang, Xueyao Wan, Shiyu Gao, Ying Ai, Hui Xu, Yongqi Hu, Hengyi Zhang, Yang Xie, Zhaxizhuoma, Yue Zhao, Zhenshan Bing, Yan Ding, Jianxing Liu

Published: 2026-06-06T08:39:52.000Z

## Abstract

Egocentric RGB-D videos offer a natural source of human dexterous manipulation demonstrations, but existing data is difficult to use for robot learning because object pose, geometry, and contact information are often missing or require pre-scanned object assets. We present EgoAERO, the first framework that learns dexterous manipulation from a single egocentric RGB-D human demonstration without object assets. EgoAERO reconstructs contact-consistent hand-object trajectories through asset-free object tracking and reconstruction, ego motion compensation, and adaptive contact optimization, then converts them into robot policies using two-stage residual learning. We further introduce an online quality assessment mechanism and construct EgoDex-R, a large-scale egocentric dataset with 4.3M RGB-D frames for dexterous policy learning. Simulation and real-world experiments show that EgoAERO enables single-demonstration dexterous manipulation and achieves downstream performance close to CAD-based reconstructions on HOI4D.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Dexterous manipulation on EgoDex-R](<https://www.sota2.com/research/sota/dexterous-manipulation-on-egodex-r.md>): Dexterous Manipulation; EgoDex-R; Object Rotation Error (Er): 9.7.

- [Dexterous Manipulation on HOI4D](<https://www.sota2.com/research/sota/dexterous-manipulation-on-hoi4d.md>): Dexterous Manipulation; HOI4D; Object Rotation Error (Er): 10.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.08057>)

[PDF](<https://arxiv.org/pdf/2606.08057.pdf>)