Driving Visual Question Answering on NuScenes QA
58.8ScoreACE-Brain 0-8B
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ACE-Brain 0-8B2026.07 | 58.8 | — | — | — | — | — | — | — | — | |
| MiMo Embodied-7B2026.07 | 56.7 | — | — | — | — | — | — | — | — | |
| ACE-Brain 0.5-8B2026.07 | 44.7 | — | — | — | — | — | — | — | — | |
| GPT 5.42026.07 | 37.1 | — | — | — | — | — | — | — | — | |
| Embodied R1.52026.07 | 33.5 | — | — | — | — | — | — | — | — | |
| RoboBrain 2.5-8B2026.07 | 33.2 | — | — | — | — | — | — | — | — | |
| Vlaser 8B2026.07 | 33.1 | — | — | — | — | — | — | — | — | |
| RoboBrain 2.0-7B2026.07 | 32.3 | — | — | — | — | — | — | — | — | |
| RynnBrain 8B2026.07 | 29.5 | — | — | — | — | — | — | — | — | |
| VeBrain 7B2026.07 | 29.3 | — | — | — | — | — | — | — | — | |
| Claude Sonnet 4.62026.07 | 22.9 | — | — | — | — | — | — | — | — | |
| Cosmos3 Nano2026.07 | 16.6 | — | — | — | — | — | — | — | — | |
| Gemini 2.5 Pro2026.07 | 16.1 | — | — | — | — | — | — | — | — | |
| Pelican VL-7B2026.07 | 14.8 | — | — | — | — | — | — | — | — | |
| BLIP-2Paradigm=Pure VLM baseline2026.05 | — | 42.3 | 60.8 | 50.4 | 20.2 | 46.1 | 38.5 | 58.2 | 36.7 | |
| DriveGPT4-styleParadigm=End-to-end V-L driving agent2026.05 | — | 52.6 | 67.3 | 69.4 | 31.2 | 58.4 | 58.6 | 76.1 | 21.5 | |
| LLaVAParadigm=Pure VLM baseline2026.05 | — | 46.1 | 63.4 | 52.1 | 21.8 | 47.9 | 41.2 | 61 | 34.6 | |
| OursParadigm=Full ICA + SSRE2026.05 | — | 64.3 | 77.5 | 76.2 | 28.9 | 56.3 | 66.4 | 93.2 | 7.4 | |
| Ours (Raw-Text)Paradigm=LLM agent w/o ICA or SSRE2026.05 | — | 49.1 | 66.2 | 60.3 | 27.1 | 52 | 50.6 | 55.1 | 41.8 | |
| Ours (w/o ICA)Paradigm=SSRE-only ablation2026.05 | — | 58.1 | 73.1 | 70.4 | 26.8 | 54 | 59.2 | 81.2 | 15.9 | |
| Ours (w/o SSRE)Paradigm=ICA-only ablation2026.05 | — | 57.3 | 72.4 | 68.2 | 26.2 | 53.2 | 57.4 | 79.3 | 17.6 | |
| PlanAgent-styleParadigm=Structured planning agent2026.05 | — | 55.7 | 70.2 | 72.6 | 28.4 | 55.6 | 63.5 | 82.4 | 46.9 | |
| Qwen2-VLParadigm=Pure VLM baseline2026.05 | — | 47.8 | 64.9 | 54.3 | 23.1 | 49.2 | 42 | 62.8 | 33.9 | |
| Talk2BEV-styleParadigm=BEV-based VLM2026.05 | — | 56.4 | 70.6 | 60.8 | 26.4 | 53.5 | 54.7 | 74.3 | 20.8 |