Question Answering on ScanQA v1.0 (val)
22.59EM@1OracleImage+MCAN (real)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| OracleImage+MCAN (real)Image Source=Oracle Image, Model=MCAN, Feature Type=real2021.12 | 22.59 | 49.43 | 26.58 | 18.32 | 15.37 | 8.5 | 33.23 | 12.45 | 63.44 | 12.56 | |
| OracleImage+Oscar (mesh)Image Source=Oracle Image, Model=Oscar, Feature Type=mesh2021.12 | 22.27 | 46.59 | 23.01 | 13.96 | 14.23 | 0 | 31.37 | 11.53 | 57.98 | 11.11 | |
| OracleImage+Oscar (real)Image Source=Oracle Image, Model=Oscar, Feature Type=real2021.12 | 21.39 | 45.05 | 24.29 | 14.19 | 14.21 | 0.67 | 31.24 | 11.33 | 58.23 | 10.51 | |
| ScanQA (multiple)Variant=multiple2021.12 | 21.05 | 51.23 | 30.24 | 20.4 | 15.11 | 10.08 | 33.33 | 13.14 | 64.86 | 13.43 | |
| OracleImage+MCAN (mesh)Image Source=Oracle Image, Model=MCAN, Feature Type=mesh2021.12 | 20.66 | 48.04 | 24.35 | 17 | 14.23 | 0 | 30.34 | 11.23 | 57.01 | 10.15 | |
| ScanQA (single)Variant=single2021.12 | 20.28 | 50.01 | 29.47 | 19.84 | 14.65 | 9.55 | 32.37 | 12.6 | 61.66 | 11.86 | |
| RandomImage+Oscar (real)Image Source=Random Image, Model=Oscar, Feature Type=real2021.12 | 19.38 | 46.37 | 22.91 | 14.52 | 11.2 | 0.56 | 28.7 | 10.66 | 52.86 | 8.91 | |
| RandomImage+MCAN (real)Image Source=Random Image, Model=MCAN, Feature Type=real2021.12 | 19.19 | 48.15 | 23.71 | 15.41 | 11.81 | 0 | 28.9 | 10.92 | 53.83 | 9.35 | |
| RandomImage+MCAN (mesh)Image Source=Random Image, Model=MCAN, Feature Type=mesh2021.12 | 18.59 | 47.81 | 22.12 | 14.49 | 11.72 | 7.69 | 27.61 | 10.32 | 50.93 | 8.37 | |
| ScanRefer+MCAN (e2e)Image Source=ScanRefer, Model=MCAN, Setup=end-to-end2021.12 | 18.59 | 46.76 | 26.93 | 16.59 | 11.59 | 7.87 | 30.03 | 11.52 | 55.41 | 11.28 | |
| RandomImage+Oscar (mesh)Image Source=Random Image, Model=Oscar, Feature Type=mesh2021.12 | 17.97 | 43.55 | 20.67 | 12.01 | 11.51 | 0.57 | 26.51 | 9.82 | 47.82 | 7.75 | |
| VoteNet+MCANImage Source=VoteNet, Model=MCAN2021.12 | 17.33 | 45.54 | 28.09 | 16.72 | 10.75 | 6.24 | 29.84 | 11.41 | 54.68 | 10.65 | |
| TopDownImage+OscarImage Source=Top-Down Image, Model=Oscar2021.12 | 17.2 | 43.81 | 19.75 | 11.21 | 15.31 | 0.71 | 25.39 | 9.43 | 45.21 | 7.32 | |
| ScanRefer+MCAN (pipeline, mesh)Image Source=ScanRefer, Model=MCAN, Setup=pipeline, Feature Type=mesh2021.12 | 14.57 | 43.27 | 16.71 | 9.71 | 13.62 | 0.64 | 21.82 | 8.32 | 38.35 | 6.49 | |
| ScanRefer+MCAN (pipeline, real)Image Source=ScanRefer, Model=MCAN, Setup=pipeline, Feature Type=real2021.12 | 14.37 | 44.12 | 17.02 | 10.17 | 15.77 | 0.72 | 22.02 | 8.45 | 38.73 | 6.66 | |
| TopDownImage+MCANImage Source=Top-Down Image, Model=MCAN2021.12 | 12.71 | 41.5 | 14.82 | 8.21 | 16.54 | 0.74 | 19.33 | 7.57 | 33.14 | 5.72 |