3D Question Answering on ScanQA w/ objects (test)
31.29EM@1BridgeQA
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| BridgeQA2024.02 | 31.29 | — | — | — | — | — | — | — | — | — | |
| BridgeQA2024.02 | 31.29 | — | 34.49 | — | — | 24.06 | 43.26 | 16.51 | 83.75 | — | |
| BridgeQAUse question to match 2D views at test time=true2024.02 | 31.19 | — | — | — | — | — | — | — | — | — | |
| 3D-VisTAObject Proposals=Mask3D, Pre-training status=Pre-trained2023.08 | 27 | 57.9 | — | — | — | 16 | 38.6 | 15.2 | 76.6 | — | |
| 3D-VisTA2024.02 | 27 | — | — | — | — | 16 | 38.6 | 15.2 | 76.6 | — | |
| 3D-VisTA2024.05 | 27 | — | 34.4 | — | — | — | — | 15.2 | 76.6 | — | |
| 3D-VisTAPre-trained=true2025.03 | 27 | 57.9 | — | — | — | 16 | 38.6 | 15.2 | 76.6 | — | |
| DSPNetPre-trained=true, LLMs-based=false, Extra dataset=true2025.03 | 26.5 | — | — | — | — | — | — | — | — | — | |
| DSPNetPre-trained=false2025.03 | 26.5 | 58.8 | — | — | — | 15.4 | 39.3 | 15.7 | 78.1 | — | |
| PQ3DTraining Protocol=unified joint training2024.05 | 26.1 | — | 43 | — | — | — | — | 17.8 | 87.8 | — | |
| PQ3DPre-trained=false, LLMs-based=false, Extra dataset=true2025.03 | 26.1 | — | — | — | — | — | — | — | — | — | |
| 3DGraphQAPre-trained=false2025.03 | 25.6 | 58.7 | — | — | — | 15.1 | 36.9 | 14.7 | 74.6 | — | |
| OracleImage+Oscar (mesh)Image Source=Oracle Image, Model=Oscar, Feature Type=mesh2021.12 | 25.52 | 52.39 | 25.17 | 14.13 | 10.94 | 0 | 33.51 | 12.68 | 63.13 | 10.52 | |
| OracleImage+MCANsampling strategy=OracleImage, 2D-QA model=MCAN2021.12 | 25.34 | 55.93 | 28.7 | 20.11 | 16.78 | 12.89 | 34.59 | 13.42 | 67.24 | 11.93 | |
| OracleImage+MCAN (real)Image Source=Oracle Image, Model=MCAN, Feature Type=real2021.12 | 25.34 | 55.93 | 28.7 | 20.11 | 16.78 | 12.89 | 34.59 | 13.42 | 67.24 | 11.93 | |
| OracleImage+Oscar (real)Image Source=Oracle Image, Model=Oscar, Feature Type=real2021.12 | 25.3 | 50.12 | 26.38 | 14.1 | 8.7 | 0.47 | 33.72 | 12.47 | 63.31 | 9.48 | |
| 3D-VisTA (scratch)Object Proposals=Mask3D, Pre-training status=From scratch2023.08 | 25.2 | 55.2 | — | — | — | 10.5 | 35.5 | 13.8 | 68.6 | — | |
| 3D-VisTAPre-trained=false2025.03 | 25.2 | 55.2 | — | — | — | 10.5 | 35.5 | 13.8 | 68.6 | — | |
| GPSPre-trained=true, LLMs-based=false, Extra dataset=true2025.03 | 25 | — | — | — | — | — | — | — | — | — | |
| 3DVLP (Jin et al. 2023)2024.02 | 24.58 | — | — | — | — | — | — | — | — | — | |
| 3DVLP2024.02 | 24.58 | — | 33.15 | — | — | 11.23 | 35.97 | 14.16 | 70.18 | — | |
| Multi-CLIP2024.02 | 24.02 | — | — | — | — | — | — | — | — | — | |
| Multi-CLIP2024.02 | 24.02 | — | 32.63 | — | — | 12.65 | 35.46 | 13.97 | 68.7 | — | |
| Multi-CLIPPre-trained=true2025.03 | 24 | — | — | — | — | 12.7 | 35.4 | 14 | 68.7 | — | |
| CLIP-Guided2024.02 | 23.92 | — | — | — | — | — | — | — | — | — | |
| CLIP-Guided2024.02 | 23.92 | — | 32.72 | — | — | 14.64 | 35.15 | 13.94 | 69.53 | — | |
| ScanQAObject Proposals=Mask3D2023.08 | 23.5 | 56.5 | — | — | — | 12 | 34.3 | 13.6 | 67.3 | — | |
| ScanQA2024.05 | 23.5 | — | 31.6 | — | — | — | — | 13.6 | 67.3 | — | |
| ScanQAPre-trained=false2025.03 | 23.5 | 56.5 | — | — | — | 12 | 34.3 | 13.6 | 67.3 | — | |
| ScanQA2021.12 | 23.45 | 56.51 | 31.56 | 21.39 | 15.87 | 12.04 | 34.34 | 13.55 | 67.29 | 11.99 | |
| ScanQA (single)Variant=single2021.12 | 23.45 | 56.51 | 31.56 | 21.39 | 15.87 | 12.04 | 34.34 | 13.55 | 67.29 | 11.99 | |
| ScanQA2024.02 | 23.45 | — | — | — | — | — | — | — | — | — | |
| ScanQA2024.02 | 23.45 | — | 31.56 | — | — | 12.04 | 34.34 | 13.55 | 67.29 | — | |
| OracleImage+MCAN (mesh)Image Source=Oracle Image, Model=MCAN, Feature Type=mesh2021.12 | 23.35 | 53.05 | 25.9 | 17.15 | 13.36 | 10.94 | 31.66 | 12.08 | 60.64 | 9.01 | |
| ScanQA (multiple)Variant=multiple2021.12 | 23.05 | 55.99 | 31.4 | 21.18 | 15.82 | 11.7 | 34.05 | 13.6 | 66.76 | 12.3 | |
| RandomImage+Oscar (real)Image Source=Random Image, Model=Oscar, Feature Type=real2021.12 | 22.65 | 52.35 | 24.74 | 14.42 | 9.85 | 0 | 30.81 | 11.59 | 57.72 | 8.52 | |
| ScanQA w/o multiview2024.02 | 22.49 | — | — | — | — | — | — | — | — | — | |
| RandomImage+MCANsampling strategy=RandomImage, 2D-QA model=MCAN2021.12 | 22.31 | 53.11 | 26.66 | 18.49 | 16.16 | 14.26 | 31.27 | 12.13 | 60.37 | 9.05 | |
| RandomImage+MCAN (real)Image Source=Random Image, Model=MCAN, Feature Type=real2021.12 | 22.31 | 53.11 | 26.66 | 18.49 | 16.16 | 14.26 | 31.27 | 12.13 | 60.37 | 9.05 | |
| Image+MCANObject Proposals=Mask3D2023.08 | 22.3 | 53.1 | — | — | — | 14.3 | 31.3 | 12.1 | 60.4 | — | |
| Image+MCANPre-trained=false2025.03 | 22.3 | 53.1 | — | — | — | 14.3 | 31.3 | 12.1 | 60.4 | — | |
| RandomImage+MCAN (mesh)Image Source=Random Image, Model=MCAN, Feature Type=mesh2021.12 | 21.74 | 52.41 | 24.86 | 17.49 | 15.33 | 15.19 | 30 | 11.55 | 57.55 | 8.73 | |
| RandomImage+Oscar (mesh)Image Source=Random Image, Model=Oscar, Feature Type=mesh2021.12 | 20.92 | 49.22 | 23.18 | 12.26 | 9.06 | 0.48 | 28.95 | 10.86 | 53.11 | 7.12 | |
| TopDownImage+OscarImage Source=Top-Down Image, Model=Oscar2021.12 | 20.76 | 49.26 | 22.19 | 11.02 | 9.3 | 0.49 | 28.25 | 10.53 | 51.82 | 6.83 | |
| ScanRefer+MCANObject Proposals=Mask3D2023.08 | 20.6 | 52.4 | — | — | — | 7.5 | 30.7 | 12 | 57.4 | — | |
| ScanRefer+MCANPre-trained=false2025.03 | 20.6 | 52.4 | — | — | — | 7.5 | 30.7 | 12 | 57.4 | — | |
| ScanRefer+MCAN (e2e)architecture=end-to-end2021.12 | 20.56 | 52.35 | 27.85 | 17.27 | 11.88 | 7.46 | 30.68 | 11.97 | 57.36 | 10.58 | |
| ScanRefer+MCAN (e2e)Image Source=ScanRefer, Model=MCAN, Setup=end-to-end2021.12 | 20.56 | 52.35 | 27.85 | 17.27 | 11.88 | 7.46 | 30.68 | 11.97 | 57.36 | 10.58 | |
| VoteNet+MCANdetector=VoteNet, 2D-QA model=MCAN2021.12 | 19.71 | 50.76 | 29.46 | 17.23 | 10.33 | 6.08 | 30.97 | 12.07 | 58.23 | 10.44 | |
| VoteNet+MCANImage Source=VoteNet, Model=MCAN2021.12 | 19.71 | 50.76 | 29.46 | 17.23 | 10.33 | 6.08 | 30.97 | 12.07 | 58.23 | 10.44 | |
| PQ3DTraining Protocol=single dataset2024.05 | 18.9 | — | 34.7 | — | — | — | — | 14.5 | 69.3 | — | |
| ScanRefer+MCAN (pipeline)architecture=pipeline2021.12 | 17.52 | 49.92 | 19.17 | 10.66 | 0 | 0 | 24.4 | 9.38 | 44.25 | 6.24 | |
| ScanRefer+MCAN (pipeline, real)Image Source=ScanRefer, Model=MCAN, Setup=pipeline, Feature Type=real2021.12 | 17.52 | 49.92 | 19.17 | 10.66 | 0 | 0 | 24.4 | 9.38 | 44.25 | 6.24 | |
| ScanRefer+MCAN (pipeline, mesh)Image Source=ScanRefer, Model=MCAN, Setup=pipeline, Feature Type=mesh2021.12 | 17.44 | 48.83 | 18.45 | 9.49 | 0 | 0 | 23.9 | 9.11 | 42.97 | 5.93 | |
| TopDownImage+MCANImage Source=Top-Down Image, Model=MCAN2021.12 | 15.76 | 47.15 | 16.52 | 8.59 | 0 | 0 | 21.49 | 8.37 | 38.55 | 5.34 |