FEVER
Benchmarks
TaskDatasetSOTA resultTrendResults
FEVER
53.9Accuracy
72
FEVER
0ASR
63
FEVER (dev)
82.1Label Accuracy
57
FEVER
98.23Efficacy
49
FEVER (val)
0.555True Deferral-Advice Loss
48
FEVER (test)
85.18Macro F1
37
FEVER 20K edits (test)
99.07Efficacy
36
FEVER
0.75Comprehensiveness
33
FEVER (test)
79.47LA Score
32
FEVER
93.6Exact Match (EM)
30
FEVER
87.45Accuracy
30
FEVER
98.38Efficacy
27
FEVER 1.0 (dev)
89.07Label Accuracy
23
FEVER BEIR
0.948nDCG
22
FEVER card (test)
63.3Accuracy (FEVER card)
21
FEVER held-out (test)
95.1Accuracy
21
FEVER Retrieved evidence
50.714SARI (%)
21
FEVER
100NDCG@5
19
FEVER BEIR
73.47NDCG@10
19
FEVER
61.1EM
18
FEVER
53.9F1 Score
18
FEVER (test)
75.96Label Accuracy (LA)
18
FEVER (test)
9.72Sufficiency
16
FEVER-Symmetric
88Precision
16
FEVER k=10 (test)
73Attack Success Rate (ASR)
15