VCR
Benchmarks
TaskDatasetSOTA resultTrendResults
VCR (val)
82.57Accuracy
63
VCR-EN-Easy
95.7EM
27
VCR-Causal
98.17Accuracy
16
VCR
68.68Overall Accuracy
15
VCR Shortcut Mitigated Evaluation SM (test)
76.32Q->A Accuracy (100%)
13
VCR Standard Evaluation (test)
61.49Q->A Accuracy (3%, 1000 SP/C)
13
Euclidean-VCR
33.2PCC > 0.9 Rate
9
VCR (val)
79.4Accuracy
8
VCR (val)
55Accuracy
8
VCR (val)
76Accuracy
8
VCR (val)
73.7Accuracy
8
VCR sampled 5000 (val)
0.84Accuracy
7
VCR MCR
36.57MUC
7
VCR (test)
51Accuracy
7
VCR 1.0 (test)
59Q->A Accuracy
7
VCR QR -> A (val)
94.7Accuracy
7
VCR 1.0 (val)
80.6Q->A Accuracy
7
VCR e-ViL (test)
26.4Meteor Score
6
VCR (test)
74Accuracy
6
VCR-Bench Img2mg
5.8Performance Score
5
VCR OC 1
36.93Accuracy
4
VCR1 MCI
60.01Accuracy
4
VCR
79.8Q->A Accuracy
4
VCR
26.4Preference Rate (Ours)
1