UltraFeedback
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
UltraFeedback clean (held-out)
0.362Kendall’s Tau
60
UltraFeedback In-domain v1 (test)
81Win Rate
46
UltraFeedback
8.1MT-Bench Score
42
UltraFeedback
30LC
42
UltraFeedback (test)
0.145MAE
38
Code-UltraFeedback
90.7Diversity
36
Ultrafeedback 61.1k (test)
69.8Win Rate
30
Ultrafeedback 61.1k (test)
73.05Accuracy
30
Ultrafeedback
86Win Rate
24
UltraFeedback (val)
88.1BERTScore
24
UltraFeedback
83Win Rate
24
UltraFeedback
68.75Accuracy
23
UltraFeedback
6.1MT-Bench Score
20
UltraFeedback
99.71Harmless Rate
20
UltraFeedback (test)
63.72Honesty Score
20
UltraFeedback Aggregate
59.59Overall Average Score
18
UltraFeedback core250
24.323Reward Score
18
UltraFeedback core250 (held-out evaluation)
3.543Delta (Δ)
18
UltraFeedback (test)
21AlpacaEval 2 Win Rate (WR)
18
UltraFeedback (UF)
7.93AOG
16
UltraFeedback (core250)
12.568Delta Preference Score (bo64)
15
UltraFeedback
0.794R2 (Top-50)
14
UltraFeedback core250
161Win Count (W)
14
UltraFeedback (test)
83.42Win Rate
14
UltraFeedback core250 (test)
80Win Rate
12