AVeriTeC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
AVeriTeC Retrieved (I) (dev)
73.6Accuracy
28
AVeriTeC Retrieved (H) (dev)
72.8Accuracy
28
AVeriTeC Golden (dev)
83.4Accuracy
28
AVeriTeC (test)
48.5Balanced Accuracy
26
AVeriTeC (dev)
46.08METEOR (Q-only)
20
AVeriTeC-DCE (test)
26.4chrF
11
AVeriTeC (test)
0.48Hu-METEOR (Q only)
9
AVeriTeC
0.3FID
8
AVERITEC
49.1Accuracy
8
AVeriTeC Retrieved (H) 50 correctly verified claims
3.67MOS
6
AVeriTeC (dev)
0.46Hu-METEOR (Q only)
6
AVeriTeC-DCE 1.0 (test)
6.6SARI
6
AVeriTeC-DCE 1.0 (test)
47.8P@1
6
AVeriTeC-DCE 1.0 (test)
6.7SARI
5
AVeriTeC (dev)
68Supported F1
4
AVeriTeC (test)
54Precision
4
AVeriTeC
77.6Macro F1 Score
3
AVERITEC
0.741Accuracy
3
AVeriTeC-DCE (full)
44IsCheckWorthy
2