3D Situated Question Answering on SQA3D (test)
54.57Average AccuracyChat-Scene
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Chat-Scene2023.12 | 54.57 | 45.38 | 67.02 | 52.04 | 69.52 | 49.85 | 54.95 | |
| Scene-LLMtask-specific fine-tuning=true2024.03 | 54.2 | 40.9 | 69.1 | 45 | 70.8 | 47.2 | 52.3 | |
| Scene-LLMVenue=ArXiv242023.12 | 54.2 | 40.9 | 69.1 | 45 | 70.8 | 47.2 | 52.3 | |
| Scene-LLMtask-specific fine-tuning=false2024.03 | 53.6 | 40 | 69.2 | 42.8 | 70.8 | 46.6 | 52.5 | |
| SIG3D2024.06 | 52.6 | 35.6 | 67.2 | 48.5 | 71.4 | 49.1 | 45.8 | |
| DSPNetPre-trained=false2025.03 | 50.4 | 38.2 | 66 | 51.2 | 66.6 | 42.5 | 51.6 | |
| 3DGraphQAPre-trained=false2025.03 | 49.2 | 36.4 | 64.7 | 46.1 | 69.8 | 47.6 | 48.2 | |
| 3D-VisTAObject Proposals=Mask3D, Pre-training=ScanScribe2023.08 | 48.5 | 34.8 | 63.3 | 45.4 | 69.8 | 47.2 | 48.1 | |
| 3D-Vistatask-specific fine-tuning=false2024.03 | 48.5 | 34.8 | 63.3 | 45.4 | 69.8 | 47.2 | 48.1 | |
| 3D-VisTA2024.06 | 48.5 | 34.8 | 63.3 | 45.4 | 69.8 | 47.2 | 48.1 | |
| 3D-VisTA2024.05 | 48.5 | 34.8 | 63.3 | 45.4 | 69.8 | 47.2 | 48.1 | |
| 3D-VisTAVenue=ICCV232023.12 | 48.5 | 34.8 | 63.3 | 45.4 | 69.8 | 47.2 | 48.1 | |
| 3D-VisTAPre-trained=true2025.03 | 48.5 | 34.8 | 63.3 | 45.4 | 69.8 | 47.2 | 48.1 | |
| LM4Vision2024.06 | 48.1 | 34.3 | 67.1 | 48.2 | 68.3 | 48.9 | 45.6 | |
| 3D-LLM2024.06 | 48.1 | 36.5 | 65.6 | 47.2 | 68.8 | 48 | 46.3 | |
| Multi-CLIP2024.06 | 48 | — | — | — | — | — | — | |
| Multi-CLIPPre-trained=true2025.03 | 48 | — | — | — | — | — | — | |
| SQA3D2024.06 | 47.2 | 33.5 | 66.1 | 42.4 | 69.5 | 43 | 46.4 | |
| SQA3DPre-trained=false2025.03 | 47.2 | 33.5 | 66.1 | 42.4 | 69.5 | 43 | 46.4 | |
| PQ3Dtraining protocol=unified training2024.05 | 47.1 | 37.1 | 61.3 | 44.5 | 60.9 | 47 | 45.1 | |
| PQ3Dtraining protocol=single-dataset2024.05 | 46.8 | 35.6 | 62.7 | 45.2 | 66.3 | 43.3 | 43.3 | |
| 3D-VisTA (scratch)Object Proposals=Mask3D, Pre-training=None2023.08 | 46.7 | 32.1 | 62.9 | 47.7 | 60.7 | 45.9 | 48.9 | |
| 3D-VisTAPre-trained=false2025.03 | 46.7 | 32.1 | 62.9 | 47.7 | 60.7 | 45.9 | 48.9 | |
| SQA3DObject Proposals=Mask3D2023.08 | 46.6 | 31.6 | 63.8 | 46 | 69.5 | 43.9 | 45.3 | |
| SQA3Dtask-specific fine-tuning=false2024.03 | 46.6 | 31.6 | 63.8 | 46 | 69.5 | 43.9 | 45.3 | |
| SQA3D2024.05 | 46.6 | 31.6 | 63.8 | 46 | 69.5 | 43.9 | 45.3 | |
| SQA3DVenue=ICLR232023.12 | 46.58 | 31.64 | 63.8 | 46.02 | 69.53 | 43.87 | 45.34 | |
| SQA3D(w/o s)Object Proposals=Mask3D2023.08 | 45.3 | 28.6 | 65 | 47.3 | 66.3 | 43.9 | 42.9 | |
| ScanQA2024.06 | 45.3 | 28.6 | 65 | 47.3 | 66.3 | 43.9 | 42.9 | |
| ScanQAPre-trained=false2025.03 | 45.3 | 28.6 | 65 | 47.3 | 66.3 | 43.9 | 42.9 | |
| MCAN2024.06 | 43.4 | 28.9 | 59.7 | 44.1 | 68.3 | 40.7 | 40.5 | |
| MCANPre-trained=false2025.03 | 43.4 | 28.9 | 59.7 | 44.1 | 68.3 | 40.7 | 40.5 | |
| ClipBERTObject Proposals=Mask3D2023.08 | 43.3 | 30.2 | 60.1 | 38.7 | 63.3 | 42.5 | 42.7 | |
| ClipBERTtask-specific fine-tuning=false2024.03 | 43.3 | 30.2 | 60.1 | 38.7 | 63.3 | 42.5 | 42.7 | |
| ClipBERT2024.06 | 43.3 | 30.2 | 60.1 | 38.7 | 63.3 | 42.5 | 42.7 | |
| ClipBERT2024.05 | 43.3 | 30.2 | 60.1 | 38.7 | 63.3 | 42.5 | 42.7 | |
| ClipBERTPre-trained=false2025.03 | 43.3 | 30.2 | 60.1 | 38.7 | 63.3 | 42.5 | 42.7 | |
| GPT-3Object Proposals=Mask3D2023.08 | 41 | 39.7 | 46 | 40.5 | 45.6 | 36.1 | 38.4 | |
| GPT-3task-specific fine-tuning=false2024.03 | 41 | 39.7 | 46 | 40.5 | 45.6 | 36.1 | 38.4 | |
| GPT-32024.06 | 41 | 39.7 | 46 | 40.5 | 45.6 | 36.1 | 38.4 |