# Differentiable Diffusion for Dense Depth Estimation from Multi-view Images

Source: [Differentiable Diffusion for Dense Depth Estimation from Multi-view Images](<https://www.sota2.com/research/paper/68fa6a441e209131cea27290>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-01T13:38:39.189Z

Authors: Numair Khan, Min H. Kim, James Tompkin

Published: 2021-06-16T16:17:34.000Z

## Abstract

We present a method to estimate dense depth by optimizing a sparse set of points such that their diffusion into a depth map minimizes a multi-view reprojection error from RGB supervision. We optimize point positions, depths, and weights with respect to the loss by differential splatting that models points as Gaussians with analytic transmittance. Further, we develop an efficient optimization routine that can simultaneously optimize the 50k+ points required for complex scene reconstruction. We validate our routine using ground truth data and show high reconstruction quality. Then, we apply this to light field and wider baseline images via self supervision, and show improvements in both average and outlier error for depth maps diffused from inaccurate sparse points. Finally, we compare qualitative and quantitative results to image processing and deep learning methods. http://visual.cs.brown.edu/diffdiffdepth

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Disparity Estimation on Living Room synthetic light field](<https://www.sota2.com/research/sota/disparity-estimation-on-living-room-synthetic-light-field.md>): Disparity Estimation; Living Room synthetic light field; MSE: 0.2.

- [Multi-view Stereo on Living Room-MVS](<https://www.sota2.com/research/sota/multi-view-stereo-on-living-room-mvs.md>): Multi-view Stereo; Living Room-MVS; MSE: 0.17.

- [Multi-view Stereo on Piano-MVS](<https://www.sota2.com/research/sota/multi-view-stereo-on-piano-mvs.md>): Multi-view Stereo; Piano-MVS; MSE: 0.69.

- [Disparity Estimation on Piano synthetic light field](<https://www.sota2.com/research/sota/disparity-estimation-on-piano-synthetic-light-field.md>): Disparity Estimation; Piano synthetic light field; Mean Squared Error (MSE): 0.0871.

- [Disparity Estimation on Dino light field (synthetic)](<https://www.sota2.com/research/sota/disparity-estimation-on-dino-light-field-synthetic.md>): Disparity Estimation; Dino light field (synthetic); MSE: 0.0086.

- [Disparity Estimation on Boxes synthetic light field](<https://www.sota2.com/research/sota/disparity-estimation-on-boxes-synthetic-light-field.md>): Disparity Estimation; Boxes synthetic light field; MSE: 0.0917.

- [Disparity Estimation on Sideboard synthetic light field](<https://www.sota2.com/research/sota/disparity-estimation-on-sideboard-synthetic-light-field.md>): Disparity Estimation; Sideboard synthetic light field; MSE: 0.0223.

- [Disparity Estimation on Cotton (synthetic light field)](<https://www.sota2.com/research/sota/disparity-estimation-on-cotton-synthetic-light-field.md>): Disparity Estimation; Cotton (synthetic light field); MSE: 0.0307.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2106.08917>)

[PDF](<https://arxiv.org/pdf/2106.08917.pdf>)

[Implementation](<https://www.sota2.com/visual.cs.brown.edu/diffdiffdepth>)