# Scan2CAD: Learning CAD Model Alignment in RGB-D Scans

Source: [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](<https://www.sota2.com/research/paper/68fa651e1e209131cea087d1>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:01:48.217Z

Authors: Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X. Chang, Matthias Nie{\\ss}ner

Published: 2018-11-27T19:00:06.000Z

## Abstract

We present Scan2CAD, a novel data-driven method that learns to align clean 3D CAD models from a shape database to the noisy and incomplete geometry of a commodity RGB-D scan. For a 3D reconstruction of an indoor scene, our method takes as input a set of CAD models, and predicts a 9DoF pose that aligns each model to the underlying scan geometry. To tackle this problem, we create a new scan-to-CAD alignment dataset based on 1506 ScanNet scans with 97607 annotated keypoint pairs between 14225 CAD models from ShapeNet and their counterpart objects in the scans. Our method selects a set of representative keypoints in a 3D scan for which we find correspondences to the CAD geometry. To this end, we design a novel 3D CNN architecture that learns a joint embedding between real and synthetic objects, and from this predicts a correspondence heatmap. Based on these correspondence heatmaps, we formulate a variational energy minimization that aligns a given set of CAD models to the reconstruction. We evaluate our approach on our newly introduced Scan2CAD benchmark where we outperform both handcrafted feature descriptor as well as state-of-the-art CNN based methods by 21.39%.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [CAD alignment on Scan2CAD (test)](<https://www.sota2.com/research/sota/cad-alignment-on-scan2cad-test.md>): CAD alignment; Scan2CAD (test); Success Rate (Bath): 37.97.

- [3D Object Model Retrieval on Scan2CAD (val)](<https://www.sota2.com/research/sota/3d-object-model-retrieval-on-scan2cad-val.md>): 3D Object Model Retrieval; Scan2CAD (val); Chamfer Distance (Chair) (mm): 2.

- [Object Alignment on Scan2CAD (val)](<https://www.sota2.com/research/sota/object-alignment-on-scan2cad-val.md>): Object Alignment; Scan2CAD (val); Chair Alignment: 44.26.

- [Object-CAD Alignment Runtime on ScanNet (val)](<https://www.sota2.com/research/sota/object-cad-alignment-runtime-on-scannet-val.md>): Object-CAD alignment; ScanNet (val); Runtime (7 Objects): 288.6.

- [CAD Alignment on Scan2CAD ScanNet v2 (val)](<https://www.sota2.com/research/sota/cad-alignment-on-scan2cad-scannet-v2-val.md>): CAD alignment; Scan2CAD ScanNet v2 (val); Alignment Score (Bath): 36.2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1811.11187>)

[PDF](<https://arxiv.org/pdf/1811.11187.pdf>)

[Implementation](<https://www.sota2.com/www.Scan2CAD.org>)
