3D Question Answering on Beacon3D
65Case ScoreMAG-3D
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MAG-3DPlanner backbone=Seed-1.6, Zero-shot=true2026.04 | 65 | 27.5 | 32.4 | 65.9 | 65.6 | 63.3 | 78 | |
| MAG-3DPlanner backbone=GPT-4o, Zero-shot=true2026.04 | 63.5 | 23.4 | 29.7 | 61.2 | 64 | 65 | 77.1 | |
| Seed-1.62026.04 | 60.2 | 23.2 | 38.9 | 65 | 54.9 | 58.2 | 67.4 | |
| LLaVA-3DData=ours, Grounded=false2025.10 | 59.1 | 21 | — | — | — | — | — | |
| SCENECOTGrounded=true2025.10 | 58.9 | 23.2 | — | — | — | — | — | |
| SCENECOTData=ours, Grounded=true2025.10 | 58.9 | 23.2 | — | — | — | — | — | |
| SceneCOT2026.04 | 58.9 | 23.2 | — | — | — | — | — | |
| GPT-4o*Grounded=false, Input=GT object labels, locations, and attributes2025.10 | 57.1 | 20.2 | — | — | — | — | — | |
| GPT-4o2026.04 | 57.1 | 20.2 | 33.3 | 49.9 | 54.9 | 52.1 | 73.8 | |
| Chat-Scene†Grounded=false, Training=Trained on paper dataset2025.10 | 53.6 | 14 | — | — | — | — | — | |
| Chat-SceneData=ours, Grounded=false2025.10 | 53.6 | 14 | — | — | — | — | — | |
| LEO†Grounded=false, Training=Trained on paper dataset2025.10 | 52.5 | 12.7 | — | — | — | — | — | |
| LEOData=ours, Grounded=false2025.10 | 52.5 | 12.7 | — | — | — | — | — | |
| MSR3D†Grounded=false, Training=Trained on paper dataset2025.10 | 51.4 | 11.9 | — | — | — | — | — | |
| MSR3DData=ours, Grounded=false2025.10 | 51.4 | 11.9 | — | — | — | — | — | |
| Chat-SceneGrounded=false, Training=SCENECOT-185K-QA(no grounded COT)2025.10 | 45.8 | 7.8 | — | — | — | — | — | |
| Chat-Scene2026.04 | 45.8 | 7.8 | 36.4 | 39.8 | 56.7 | 47.6 | 48.8 | |
| LEOGrounded=false, Training=SCENECOT-185K-QA(no grounded COT)2025.10 | 43.2 | 7.8 | — | — | — | — | — | |
| LEO2026.04 | 43.2 | 7.8 | 17.4 | 41 | 53.2 | 48.7 | 39.7 | |
| SceneVerse2026.04 | 40.3 | 6.6 | 35.6 | 41.7 | 48.9 | 41.9 | 35.7 | |
| 3D-VisTA2026.04 | 35.3 | 8.1 | 20.5 | 33.5 | 52.1 | 33.8 | 36.5 | |
| PQ3D-LLM2026.04 | 27.9 | 2.3 | 28 | 30.8 | 35.2 | 25.2 | 26.2 | |
| PQ3D2026.04 | 27.8 | 3.5 | 36.4 | 28 | 27.8 | 11.9 | 45.5 |