VQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
VQA ITA
48Accuracy
6
VQA Shadow
0.67Accuracy
6
VQA Natural Light
68Accuracy
6
VQA Clean
68Accuracy
6
VQA-Rephrasings (Near OOD)
79.48Accuracy
6
VQA v2 (ID)
87.39Accuracy
6
VQA Partial Train setup (val)
80.2In-domain Accuracy
6
VQA Efficiency (450frs@1080p) (test)
79.812Average Computation Time (s)
6
VQA Efficiency 364frs@480p (test)
12.341Average Computation Time (s)
6
VQA v2
0.853AUROC
5
VQA v2
0Refusal Rate (Exact)
5
VQA v2
81.2Accuracy
5
VQA v2
73.2Accuracy (clean)
5
VQA Text
79.65Exact Match
5
VQA Reliable Challenge v2 (test-standard)
84.11Accuracy
5
VQA^L
84.3Accuracy
5
VQA (test-val)
76.48Accuracy
5
VQA v2
57.72Accuracy (Rec.)
4
VQA v2
100Exact Match
4
VQA 2.0
54.5Accuracy
4
VQA v2
78.5Accuracy
4
VQA v2
78.7Accuracy
4
VQA (val)
79.99RAD (Y/N ← C)
4
VQA v2
80.2Accuracy
4
VQA v2
77.8Accuracy
4