FEVER
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
FEVER (dev)
76.3Label Accuracy (LA)
9
FEVER (test)
0.796NDCG@10
9
FEVER
0.737AUROC
8
FEVER
61Precision (macro)
8
FEVER S R
95.2Precision
8
FEVER leaderboard March 2019 (test)
77.7Evidence F1
8
FEVER frozen 100-client instance (held-out)
55Accuracy
7
FEVER
0.132Utility Score
7
FEVER
75.3Exact Match (EM)
7
FEVER (test)
89.38Accuracy
7
FEVER
99.7Accuracy
7
FEVER
58.42Entity in Context
7
FEVER
77Hit Rate
6
FEVER n=200
1M4
6
FEVER
0.861Accuracy
6
FEVER (blind test)
76.85Label Accuracy
6
Large FEVER Extended (val)
18.69Utility
5
FEVER
50.8Relevance
5
FEVER out-of-domain (test)
77.6nDCG@10
5
FEVER
0.915NDCG@10
5
FEVER
0.961TP Rate
5
FEVER
68.7AUC
5
FEVER 2.0
49.99Recall@3
5
FEVER ERASER (test)
0.9Weighted Avg F1
5
FEVER Loose-Multi
77.9LA
5