3D Question Answering on MSQA
47.9Count AccuracySCENECOT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| SCENECOTGrounded=true2025.10 | 47.9 | 82.1 | 49.6 | 47.2 | 51.6 | 80.3 | 55.6 | |
| SCENECOTData=ours, Grounded=true2025.10 | 47.7 | 82.1 | 49.6 | 47.2 | 51.6 | 80.3 | 55.6 | |
| MAG-3DSeed-1.6Input=T+I+O2026.04 | 44.1 | 88.5 | 62.2 | 33.9 | 53.5 | 73.7 | 54.3 | |
| LLaVA-3DData=ours, Grounded=false2025.10 | 38.5 | 89.5 | 57 | 49.8 | 38.5 | 84.4 | 54.9 | |
| Chat-Scene†Grounded=false, Training=Trained on paper dataset2025.10 | 37.4 | 92 | 49 | 47 | 58.3 | 83.7 | 56.6 | |
| Chat-SceneData=ours, Grounded=false2025.10 | 37.4 | 92 | 49 | 47 | 58.3 | 83.7 | 56.6 | |
| MAG-3DGPT-4oInput=T+I+O2026.04 | 37 | 85.6 | 66.5 | 31.2 | 53.2 | 79.2 | 53 | |
| GPT-3.5Input=T+O2026.04 | 34.8 | 74.5 | 75.8 | 27.9 | 43 | 88 | 50.7 | |
| Loc3R-VLM2026.03 | 33.1 | 88.5 | 61.3 | 57.6 | 47.2 | 83.8 | 58.6 | |
| MSR3D†Grounded=false, Training=Trained on paper dataset2025.10 | 32.7 | 87.5 | 53.7 | 44.3 | 51.5 | 72.3 | 52.8 | |
| MSR3DData=ours, Grounded=false2025.10 | 32.7 | 87.5 | 53.7 | 44.3 | 51.5 | 72.3 | 52.8 | |
| Claude-3.5-SonnetInput=T+I+O2026.04 | 32.6 | 66.3 | 69.9 | 30.1 | 45.5 | 83.6 | 49.7 | |
| LEOGrounded=false, Training=SCENECOT-185K-QA(no grounded COT)2025.10 | 32.5 | 88.5 | 58.7 | 44.2 | 39.6 | 81.4 | 54.8 | |
| LEO2026.03 | 32.5 | 88.5 | 58.7 | 44.2 | 39.6 | 81.4 | 54.8 | |
| GPT-4o*Grounded=false, Input=GT object labels, locations, and attributes2025.10 | 32.3 | 79.3 | 79 | 37 | 31.7 | 91.6 | 52.3 | |
| MSR3DGrounded=false, Training=SCENECOT-185K-QA(no grounded COT)2025.10 | 32.3 | 93.1 | 50 | 46.5 | 54.1 | 75.6 | 54.2 | |
| GPT-4o2026.03 | 32.3 | 79.3 | 79 | 37 | 31.7 | 91.6 | 52.3 | |
| MSR3DGround-truth location input=true2026.03 | 32.3 | 93.1 | 50 | 46.5 | 54.1 | 75.6 | 54.2 | |
| Seed-1.6Input=T+I+O2026.04 | 31.3 | 75.1 | 69.4 | 36.8 | 24 | 77.1 | 47.9 | |
| GPT-4oInput=T+I+O2026.04 | 31.2 | 71.4 | 75.2 | 31.5 | 36.7 | 88 | 49.7 | |
| LEO†Grounded=false, Training=Trained on paper dataset2025.10 | 29.3 | 87.5 | 55.1 | 46.3 | 45.6 | 81.4 | 52.9 | |
| LEOData=ours, Grounded=false2025.10 | 29.3 | 87.5 | 55.1 | 46.3 | 45.6 | 81.4 | 52.9 | |
| SplatTalk‡Grounded=false, Data Version=Not based on Version-2.12025.10 | 19.6 | 60.3 | 44 | 35.8 | 35.5 | 61.8 | 41.8 | |
| SplatTalk2026.03 | 19.6 | 60.3 | 44 | 35.8 | 35.5 | 61.8 | 41.8 | |
| LEOInput=T+PC2026.04 | 0.8 | 15.5 | 11.8 | 7.3 | 2.3 | 15.3 | 7.8 |