# Evaluating Object (mis)Detection from a Safety and Reliability Perspective: Discussion and Measures

Source: [Evaluating Object (mis)Detection from a Safety and Reliability Perspective: Discussion and Measures](<https://www.sota2.com/research/paper/68fa713e1e209131cea5008b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:01:40.608Z

Authors: Andrea Ceccarelli, Leonardo Montecchi

Published: 2022-03-04T09:31:20.000Z

## Abstract

We argue that object detectors in the safety critical domain should prioritize detection of objects that are most likely to interfere with the actions of the autonomous actor. Especially, this applies to objects that can impact the actor's safety and reliability. To quantify the impact of object (mis)detection on safety and reliability in the context of autonomous driving, we propose new object detection measures that reward the correct identification of objects that are most dangerous and most likely to affect driving decisions. To achieve this, we build an object criticality model to reward the detection of the objects based on proximity, orientation, and relative velocity with respect to the subject vehicle. Then, we apply our model on the recent autonomous driving dataset nuScenes, and we compare nine object detectors. Results show that, in several settings, object detectors that perform best according to the nuScenes ranking are not the preferable ones when the focus is shifted on safety and reliability.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2203.02205>)

[PDF](<https://arxiv.org/pdf/2203.02205.pdf>)
