# Criticality Metrics for Relevance Classification in Safety Evaluation of Object Detection in Automated Driving

Source: [Criticality Metrics for Relevance Classification in Safety Evaluation of Object Detection in Automated Driving](<https://www.sota2.com/research/paper/69438a72423ea4226572937c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-19T05:00:04.527Z

Authors: J\\"org Gamerdinger, Sven Teufel, Stephan Amann, Oliver Bringmann

Published: 2025-12-17T08:28:53.000Z

## Abstract

Ensuring safety is the primary objective of automated driving, which necessitates a comprehensive and accurate perception of the environment. While numerous performance evaluation metrics exist for assessing perception capabilities, incorporating safety-specific metrics is essential to reliably evaluate object detection systems. A key component for safety evaluation is the ability to distinguish between relevant and non-relevant objects - a challenge addressed by criticality or relevance metrics. This paper presents the first in-depth analysis of criticality metrics for safety evaluation of object detection systems. Through a comprehensive review of existing literature, we identify and assess a range of applicable metrics. Their effectiveness is empirically validated using the DeepAccident dataset, which features a variety of safety-critical scenarios. To enhance evaluation accuracy, we propose two novel application strategies: bidirectional criticality rating and multi-metric aggregation. Our approach demonstrates up to a 100% improvement in terms of criticality classification accuracy, highlighting its potential to significantly advance the safety evaluation of object detection systems in automated vehicles.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Criticality Classification on DeepAccident (test)](<https://www.sota2.com/research/sota/criticality-classification-on-deepaccident-test.md>): Criticality Classification; DeepAccident (test); Safety Rate (SR): 100.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2512.15181>)

[PDF](<https://arxiv.org/pdf/2512.15181.pdf>)