Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset
About
6D object pose estimation is one of the fundamental problems in computer vision and robotics research. While a lot of recent efforts have been made on generalizing pose estimation to novel object instances within the same category, namely category-level 6D pose estimation, it is still restricted in constrained environments given the limited number of annotated data. In this paper, we collect Wild6D, a new unlabeled RGBD object video dataset with diverse instances and backgrounds. We utilize this data to generalize category-level 6D object pose estimation in the wild with semi-supervised learning. We propose a new model, called Rendering for Pose estimation network RePoNet, that is jointly trained using the free ground-truths with the synthetic data, and a silhouette matching objective function on the real-world data. Without using any 3D annotations on real data, our method outperforms state-of-the-art methods on the previous dataset and our Wild6D test set (with manual annotations for evaluation) by a large margin. Project page with Wild6D data: https://oasisyang.github.io/semi-pose .
Key insights
How does semi-supervised learning advance category-level pose estimation?
The paper estimates 6D pose (rotation, translation, size) for unseen objects within a category. It introduces Wild6D, an unlabeled RGBD video dataset, and RePoNet, which trains on synthetic data with full annotations and real data using only silhouette matching via differentiable rendering.
What distinguishes Wild6D and RePoNet from prior work?
Wild6D provides 5,166 videos (over 1.1 million images) across 1,722 instances, roughly 300x larger than REAL275. RePoNet-semi outperforms CPS++ on REAL275 and achieves 29.5% on Wild6D at 5 degree, 2cm, versus Shape-Prior at 2.6%.
What are the practical uses of these pose estimation findings?
Wild6D supplies diverse objects and backgrounds for testing pose estimation beyond controlled lab imagery. RePoNet shows how synthetic labels and unlabeled real video can be combined, while the released dataset supports comparison of future methods.
What limitations does RePoNet have regarding generalization?
RePoNet depends on predefined category-level mesh priors, making the learned deformation model fixed to a specific category and hard to generalize to unseen categories. The authors suggest representing object shape via unit sphere deformation as an alternative.