TruthfulQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
TruthfulQA
64.5ROUGE-1
4
TruthfulQA
55.6Accuracy
4
TruthfulQA (test)
0.031Steering Effect
4
TruthfulQA
81.5Truthfulness x Informativeness Score
4
TruthfulQA
55.6Accuracy
4
TruthfulQA (full)
0.599AUROC
4
TruthfulQA
59.9AUROC
4
TruthfulQA DE
36.2Normalized Probability Mass
4
TruthfulQA
36.4Normalized Probability Mass
4
TruthfulQA
55.6Factuality Score
4
TruthfulQA MC2 0-shot (test)
64.27Accuracy
4
TruthfulQA (TQA)
66.4Accuracy (TQA)
4
TruthfulQA OOD
52.2Wins
4
TruthfulQA
1.674BCE
4
TruthfulQA (train test)
0.523BLEU
4
Translated TruthfulQA (test)
81.4Accuracy - English
4
TruthfulQA
1.96MAE
3
TruthfulQA
0.605Kendall's Tau Correlation
3
TruthfulQA
0.858AUROC
3
TruthfulQA (val)
0.906AUROC
3
TruthfulQA
188A Wins
3
TruthfulQA generation
7.9Exclusive Catch Rate (@10%)
3
TruthfulQA (adversarial)
1Sycophantic Response Count
3
TruthfulQA Adversarial n=50
2Sycophantic Responses Count
3
TruthfulQA 200-question subset
0.512AUROC
3