# CLIP-BEVFormer: Enhancing Multi-View Image-Based BEV Detector with Ground Truth Flow

Source: [CLIP-BEVFormer: Enhancing Multi-View Image-Based BEV Detector with Ground Truth Flow](<https://www.sota2.com/research/paper/68fa75d01e209131cea69fc4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T08:08:46.110Z

Authors: Chenbin Pan, Burhaneddin Yaman, Senem Velipasalar, Liu Ren

Published: 2024-03-13T19:21:03.000Z

## Abstract

Autonomous driving stands as a pivotal domain in computer vision, shaping the future of transportation. Within this paradigm, the backbone of the system plays a crucial role in interpreting the complex environment. However, a notable challenge has been the loss of clear supervision when it comes to Bird's Eye View elements. To address this limitation, we introduce CLIP-BEVFormer, a novel approach that leverages the power of contrastive learning techniques to enhance the multi-view image-derived BEV backbones with ground truth information flow. We conduct extensive experiments on the challenging nuScenes dataset and showcase significant and consistent improvements over the SOTA. Specifically, CLIP-BEVFormer achieves an impressive 8.5\\% and 9.2\\% enhancement in terms of NDS and mAP, respectively, over the previous best BEV model on the 3D object detection task.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [3D Object Detection on nuScenes (val)](<https://www.sota2.com/research/sota/3d-object-detection-on-nuscenes-val.md>): 3D Object Detection; nuScenes (val); NDS: 56.2.

- [3D Object Detection on nuScenes (test)](<https://www.sota2.com/research/sota/3d-object-detection-on-nuscenes-test.md>): 3D Object Detection; nuScenes (test); mAP: 44.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2403.08919>)

[PDF](<https://arxiv.org/pdf/2403.08919.pdf>)
