ScanQA
Benchmarks
TaskDatasetSOTA resultTrendResults
ScanQA (val)
107.1CIDEr
391
ScanQA w/ objects (test)
31.29EM@1
55
ScanQA w/o objects (test)
30.87EM@1
51
ScanQA
30.3EM (Exact Match)
48
ScanQA
15.9B-4 Score
31
ScanQA v1.0 (test)
23BLEU-4
28
ScanQA (test)
13.9BLEU-4
20
ScanQA
30.1EM@1
17
ScanQA
20.8METEOR
16
ScanQA
88.77CIDEr
16
ScanQA v1.0 (val)
22.59EM@1
16
ScanQA (evaluation)
107.4CIDEr
13
ScanQA ScanNet scenes (test)
43.3BLEU-1
13
ScanQA v1.0 (val)
16.2BLEU-4
13
ScanQA
22.9M Score
6
ScanQA Leaderboard as of Nov 2023 (test)
30.82EM
5
ScanQA
30.5EM
4
ScanQA
40.8F1@50
3
ScanQA
24.5EM-R
3
ScanQA w/ objects (test)
25.44Acc@0.25
3
ScanQA (val)
24.96Acc@0.25
3
ScanQA-G
40.8F1@50
2