# Mono2Stereo: Monocular Knowledge Transfer for Enhanced Stereo Matching

Source: [Mono2Stereo: Monocular Knowledge Transfer for Enhanced Stereo Matching](<https://www.sota2.com/research/paper/68fa75af1e209131cea69480>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:39:44.122Z

Authors: Yuran Wang, Yingping Liang, Hesong Li, Ying Fu

Published: 2024-11-14T03:01:36.000Z

## Abstract

The generalization and performance of stereo matching networks are limited due to the domain gap of the existing synthetic datasets and the sparseness of GT labels in the real datasets. In contrast, monocular depth estimation has achieved significant advancements, benefiting from large-scale depth datasets and self-supervised strategies. To bridge the performance gap between monocular depth estimation and stereo matching, we propose leveraging monocular knowledge transfer to enhance stereo matching, namely Mono2Stereo. We introduce knowledge transfer with a two-stage training process, comprising synthetic data pre-training and real-world data fine-tuning. In the pre-training stage, we design a data generation pipeline that synthesizes stereo training data from monocular images. This pipeline utilizes monocular depth for warping and novel view synthesis and employs our proposed Edge-Aware (EA) inpainting module to fill in missing contents in the generated images. In the fine-tuning stage, we introduce a Sparse-to-Dense Knowledge Distillation (S2DKD) strategy encouraging the distributions of predictions to align with dense monocular depths. This strategy mitigates issues with edge blurring in sparse real-world labels and enhances overall consistency. Experimental results demonstrate that our pre-trained model exhibits strong zero-shot generalization capabilities. Furthermore, domain-specific fine-tuning using our pre-trained model and S2DKD strategy significantly increments in-domain performance. The code will be made available soon.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Stereo Conversion on Inria 3D Movie dataset](<https://www.sota2.com/research/sota/stereo-conversion-on-inria-3d-movie-dataset.md>): Stereo Conversion; Inria 3D Movie dataset; RMSE: 6.857.

- [Stereo Matching on KITTI (D1 Error Breakdown)](<https://www.sota2.com/research/sota/stereo-matching-on-kitti-d1-error-breakdown.md>): Stereo Matching; KITTI; D1 Error (Non-occ): 1.48.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2411.09151>)

[PDF](<https://arxiv.org/pdf/2411.09151.pdf>)
