# Deep Laparoscopic Stereo Matching with Transformers

Source: [Deep Laparoscopic Stereo Matching with Transformers](<https://www.sota2.com/research/paper/68fa6d161e209131cea37939>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:17:58.959Z

Authors: Xuelian Cheng, Yiran Zhong, Mehrtash Harandi, Tom Drummond, Zhiyong Wang, Zongyuan Ge

Published: 2022-07-25T12:54:32.000Z

## Abstract

The self-attention mechanism, successfully employed with the transformer structure is shown promise in many computer vision tasks including image recognition, and object detection. Despite the surge, the use of the transformer for the problem of stereo matching remains relatively unexplored. In this paper, we comprehensively investigate the use of the transformer for the problem of stereo matching, especially for laparoscopic videos, and propose a new hybrid deep stereo matching framework (HybridStereoNet) that combines the best of the CNN and the transformer in a unified design. To be specific, we investigate several ways to introduce transformers to volumetric stereo matching pipelines by analyzing the loss landscape of the designs and in-domain/cross-domain accuracy. Our analysis suggests that employing transformers for feature representation learning, while using CNNs for cost aggregation will lead to faster convergence, higher accuracy and better generalization than other options. Our extensive experiments on Sceneflow, SCARED2019 and dVPN datasets demonstrate the superior performance of our HybridStereoNet.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Stereo Matching on dVPN (test)](<https://www.sota2.com/research/sota/stereo-matching-on-dvpn-test.md>): Stereo Matching; dVPN (test); Mean SSIM: 56.98.

- [Stereo Matching on SCARED reorganized (test-19)](<https://www.sota2.com/research/sota/stereo-matching-on-scared-reorganized-test-19.md>): Stereo Matching; SCARED reorganized (test-19); EPE (px): 1.4096.

- [Stereo Matching on SCARED Set 1 Original 2019 (test)](<https://www.sota2.com/research/sota/stereo-matching-on-scared-set-1-original-2019-test.md>): Stereo Matching; SCARED Set 1 Original 2019 (test); Error kf1: 7.96.

- [Stereo Matching on SCARED Set 2 Original 2019 (test)](<https://www.sota2.com/research/sota/stereo-matching-on-scared-set-2-original-2019-test.md>): Stereo Matching; SCARED Set 2 Original 2019 (test); KF1 Score: 4.57.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2207.12152>)

[PDF](<https://arxiv.org/pdf/2207.12152.pdf>)

[Implementation](<https://mmai.group/>)
