HaluEval
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
HaluEval QA (test)
78.9TPR
8
HaluEval Summarization (Starling-LM-7B-alpha)
81TPR
7
HaluEval
31Accuracy
6
HaluEval Summarization
50Accuracy
6
HaluEval QAmis (test)
0.008Failure Rate
6
HaluEval (test)
0.36Failure Rate (Sum)
6
HaluEval qa_samples
86.7F1 Score
5
HaluEval QA
69.45Accuracy
5
HaluEval
42.4Macro F1
4
HaluEval
69Accuracy
4
HaluEval Amharic - Low (test)
78.5AUROC
4
HaluEval Bangla - Medium (test)
89.1AUROC
4
HaluEval French - High (test)
96.2AUROC (%)
4
HaluEval English - Base (test)
98.5AUROC
4
HaluEval
11.3nAUPC
4
HaluEval QA 60 (test)
0.971AUROC
3
HaluEval General
30Baseline Wins
2
HaluEval (random sample of 1,000 text pairs)
95.3Recall
1
HaluEval
27.9Accuracy
1
HaluEval Dialogue (test)
0.287Groundedness (Gamma)
1
HaluEval ChatGPT (test)
94.5Coverage
1