# On Offline Evaluation of 3D Object Detection for Autonomous Driving

Source: [On Offline Evaluation of 3D Object Detection for Autonomous Driving](<https://www.sota2.com/research/paper/68fa705f1e209131cea4aaf5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:40:08.455Z

Authors: Tim Schreier, Katrin Renz, Andreas Geiger, Kashyap Chitta

Published: 2023-08-24T13:31:51.000Z

## Abstract

Prior work in 3D object detection evaluates models using offline metrics like average precision since closed-loop online evaluation on the downstream driving task is costly. However, it is unclear how indicative offline results are of driving performance. In this work, we perform the first empirical evaluation measuring how predictive different detection metrics are of driving performance when detectors are integrated into a full self-driving stack. We conduct extensive experiments on urban driving in the CARLA simulator using 16 object detection models. We find that the nuScenes Detection Score has a higher correlation to driving performance than the widely used average precision metric. In addition, our results call for caution on the exclusive reliance on the emerging class of \`planner-centric' metrics.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2308.12779>)

[PDF](<https://arxiv.org/pdf/2308.12779.pdf>)