Autonomous Driving Model Testing on CARLA SafeBench (Multiple Regions)
93.3Compliance Ratio (CR)TRAFFICALIGN
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TRAFFICALIGNRegion=Los Angeles, Driving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 93.3 | 31.6 | 28.3 | 9.1 | 71.3 | 52.4 | 49.1 | 76.6 | 60.1 | 31.9 | 40.5 | |
| TRAFFICALIGNRegion=New York City, Driving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 92.3 | 46.6 | 28.1 | 3.7 | 70 | 52.7 | 39.1 | 79.3 | 54.1 | 35.7 | 31.9 | |
| TRAFFICALIGNRegion=Switzerland, Driving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 91.9 | 37.9 | 20.1 | 4.1 | 72.3 | 51.4 | 38.3 | 76.1 | 51.4 | 39.3 | 42.9 | |
| TRAFFICALIGNRegion=Yellowstone, Driving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 90.9 | 45.6 | 22.1 | 10.4 | 70.8 | 43.8 | 29.8 | 80.9 | 47.6 | 35.5 | 31 | |
| TRAFFICALIGNRegion=Yosemite, Driving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 90.3 | 42 | 17.9 | 3.9 | 74 | 58.2 | 47.7 | 75.6 | 66 | 21.8 | 45.7 | |
| TRAFFICALIGNRegion=Small Towns, Driving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 85.2 | 18.6 | 14.9 | 3.8 | 79.6 | 56.1 | 26.6 | 73.5 | 51.7 | 28.3 | 43.7 | |
| ChatSceneDriving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 82.5 | 19.1 | 14.2 | 3.5 | 86.1 | 53.7 | 21.3 | 70.6 | 53.3 | 22.8 | 48.1 | |
| Carla Scenario GeneratorDriving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 67.6 | 31.3 | 16.1 | 3.6 | 89 | 74.1 | 24.4 | 21.5 | 24.3 | 13.1 | 57.3 | |
| Adv. Trajectory Optim.Driving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 62.7 | 31.2 | 15.8 | 2.8 | 89.3 | 72.6 | 27.9 | 21.9 | 24.8 | 13.7 | 59.6 | |
| AdvSimDriving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 58.6 | 30 | 16 | 2.5 | 89.1 | 74.5 | 26.1 | 20.3 | 24.5 | 12.7 | 62 | |
| Learning-to-collideDriving Model Evaluation Protocol=Average of PPO, SAC, and TD32026.06 | 58.4 | 32.6 | 15.8 | 3.2 | 89.4 | 73.1 | 21.6 | 21.1 | 24.3 | 11.2 | 61.9 |