# Revealing Occlusions with 4D Neural Fields

Source: [Revealing Occlusions with 4D Neural Fields](<https://www.sota2.com/research/paper/68fa6c601e209131cea33aa8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:14:46.223Z

Authors: Basile Van Hoorick, Purva Tendulkar, Didac Suris, Dennis Park, Simon Stent, Carl Vondrick

Published: 2022-04-22T20:14:42.000Z

## Abstract

For computer vision systems to operate in dynamic situations, they need to be able to represent and reason about object permanence. We introduce a framework for learning to estimate 4D visual representations from monocular RGB-D, which is able to persist objects, even once they become obstructed by occlusions. Unlike traditional video representations, we encode point clouds into a continuous representation, which permits the model to attend across the spatiotemporal context to resolve occlusions. On two large video datasets that we release along with this paper, our experiments show that the representation is able to successfully reveal occlusions for several tasks, without any architectural changes. Visualizations show that the attention mechanism automatically learns to follow occluded objects. Since our approach can be trained end-to-end and is easily adaptable, we believe it will be useful for handling occlusions in many video understanding tasks. Data, code, and models are available at https://occlusions.cs.columbia.edu/.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Geometry Completion on CARLA](<https://www.sota2.com/research/sota/geometry-completion-on-carla.md>): Geometry Completion; CARLA; Chamfer Distance (All): 0.47.

- [Geometry Completion on GREATER](<https://www.sota2.com/research/sota/geometry-completion-on-greater.md>): Geometry Completion; GREATER; Chamfer Distance (All): 0.22.

- [Instance Tracking on GREATER](<https://www.sota2.com/research/sota/instance-tracking-on-greater.md>): Instance Tracking; GREATER; Chamfer Distance (Avg. Inst.): 1.05.

- [Semantic Segmentation on CARLA](<https://www.sota2.com/research/sota/semantic-segmentation-on-carla.md>): Semantic segmentation; CARLA; CD (Avg Class): 5.82.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2204.10916>)

[PDF](<https://arxiv.org/pdf/2204.10916.pdf>)

[Implementation](<https://www.sota2.com/occlusions.cs.columbia.edu>)