CARLA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
CARLA (All episodes)
0.334Distance to Centerline (m)
3
CARLA Near-Lane Obstacle Exp. 2
0Completion Rate (CR)
3
CARLA Exp. 1 (In-Lane Obstacle)
100Completion Rate (CR)
3
CARLA generated scenarios
25.73Colorfulness
3
CARLA-based generated scenarios
10.392Mean Velocity
3
CARLA Town02 (unseen)
0.71Mean Return
3
CARLA Traffic Density 1.0
0.006Safety
3
CARLA Traffic Density 0.75
0.006Safety Deviation
3
CARLA Traffic Density 0.5
0.004Safety
3
CARLA Dense traffic
17.24AS Score
3
CARLA Regular traffic
17.47AS
3
CARLA Empty traffic
17.81AS Score
3
CARLA Town 5
17.75AS
3
CARLA Town 2
17.47AS
3
CARLA Town 1
17.91Average Score (AS)
3
CARLA intersection environments Traffic Density 1.0 (hold-out)
56Success Rate
3
CARLA hold-out intersection environments (Traffic Density 0.75)
68Success Rate
3
CARLA hold-out intersection environments Traffic Density 0.5
80Success Rate
3
CARLA Town02 (unseen)
75SR (Straight)
3
CARLA Town01 (seen)
76.5Straight Success Rate
3
CARLA Long-tail Semantic Failures (in-distribution)
91.8Emergency Vehicle Accuracy (Cat 1)
3
CARLA semantic-failure scenarios
100Emergency Vehicle Recall
3
CARLA Simulation 3
8.658Registration Score
3
CARLA Simulation 2
5.252Registration Error
3
CARLA Simulation Sim 1
2.715Registration Score
3