Closed-loop Autonomous Driving Evaluation on Waymo
0False DetectionsWaymo
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| WaymoNPC Policy=Replay, Ego Planner=IDM (Rule-based), Online=false, Warm=false2026.03 | 0 | — | 7.2 | 5.8 | 87 | 64 | 11.4 | — | — | |
| WaymoNPC Policy=Replay, Ego Planner=PPO (Pre-trained), Online=false, Warm=false2026.03 | 0 | — | 29.3 | 6.9 | 63.8 | 200 | — | 36.2 | — | |
| VectorWorldNPC Policy=ΔSim(Pos.), Ego Planner=PPO (Pre-trained), Online=true, Warm=true2026.03 | 0.92 | 2.13 | 30.7 | 7.2 | 62.3 | 1,000 | — | 37.9 | — | |
| VectorWorldNPC Policy=ΔSim, Ego Planner=IDM (Rule-based), Online=true, Warm=true2026.03 | 0.94 | 2.95 | 41.6 | 7 | 51.4 | 1,000 | 9.6 | — | — | |
| VectorWorldNPC Policy=ΔSim, Ego Planner=IDM (Rule-based), Online=false, Warm=false2026.03 | 1.05 | 4.69 | 7.5 | 1.5 | 91 | 200 | 10.4 | — | — | |
| VectorWorldNPC Policy=ΔSim(Neg.), Ego Planner=PPO (Pre-trained), Online=true, Warm=true2026.03 | 1.15 | 8.55 | 65.1 | 9.2 | 25.7 | 1,000 | — | 75 | — | |
| VectorWorldNPC Policy=ΔSim(Neg.), Ego Planner=PPO (Pre-trained), Online=true, Warm=true, Reason=Closed-Loop Training Section2026.03 | 1.15 | 8.55 | 65.1 | 9.2 | 25.7 | 1,000 | — | — | 25.7 | |
| VectorWorldNPC Policy=ΔSim(Neg.), Ego Planner=PPO (Retrained), Online=true, Warm=true2026.03 | 1.15 | 8.55 | 35.1 | 7.9 | 56 | 1,000 | — | — | 56 | |
| VectorWorldNPC Policy=ΔSim, Ego Planner=IDM (Rule-based), Online=true, Warm=false2026.03 | 1.19 | 7.39 | 45.5 | 6.9 | 47.6 | 1,000 | 16.6 | — | — | |
| ScenDream-LNPC Policy=Ctrl-sim, Ego Planner=IDM (Rule-based), Online=false, Warm=false2026.03 | 1.38 | 4.8 | 12.4 | 5.9 | 81.7 | 200 | 15.5 | — | — | |
| ScenDream-LNPC Policy=Ctrl-sim (Pos.), Ego Planner=PPO (Pre-trained), Online=false, Warm=false2026.03 | 1.42 | 5.1 | 52.8 | 9.1 | 38.2 | 200 | — | 61.8 | — | |
| ScenDream-LNPC Policy=Ctrl-sim (Neg.), Ego Planner=PPO (Pre-trained), Online=false, Warm=false2026.03 | 1.42 | 11.7 | 59 | 9 | 32.1 | 200 | — | 67.9 | — |