AVeriTeC
Benchmarks
TaskDatasetSOTA resultTrendResults
AVeriTeC Retrieved (I) (dev)
73.6Accuracy
28
AVeriTeC Retrieved (H) (dev)
72.8Accuracy
28
AVeriTeC Golden (dev)
83.4Accuracy
28
AVeriTeC (test)
48.5Balanced Accuracy
26
AVeriTeC (dev)
46.08METEOR (Q-only)
20
AVeriTeC-DCE (test)
26.4chrF
11
AVeriTeC (test)
0.48Hu-METEOR (Q only)
9
AVeriTeC
0.3FID
8
AVERITEC
49.1Accuracy
8
AVeriTeC Retrieved (H) 50 correctly verified claims
3.67MOS
6
AVeriTeC (dev)
0.46Hu-METEOR (Q only)
6
AVeriTeC-DCE 1.0 (test)
6.6SARI
6
AVeriTeC-DCE 1.0 (test)
47.8P@1
6
AVeriTeC-DCE 1.0 (test)
6.7SARI
5
AVeriTeC (dev)
68Supported F1
4
AVeriTeC (test)
54Precision
4
AVeriTeC
77.6Macro F1 Score
3
AVERITEC
0.741Accuracy
3
AVeriTeC-DCE (full)
44IsCheckWorthy
2