UltraFeedback
Benchmarks
TaskDatasetSOTA resultTrendResults
UltraFeedback
80.8Win Rate
4
UltraFeedback 2023 (test)
55Win-rate
4
UltraFeedback Format-Biased (test)
89Win-Rate (Bold)
4
UltraFeedback 13B+ Models Holdout (test)
74.8Pairwise Accuracy (RM1_Honest)
4
UltraFeedback 30B+ Models Holdout (test)
77.3Pairwise Acc (RM1_Honest)
4
UltraFeedback 70B+ Models Holdout (test)
77.4Pairwise Acc (RM1_Honest)
4
UltraFeedback 10% holdout (test)
86.3Pairwise Accuracy (RM1-Honest)
4
UltraFeedback 5% holdout (test)
87Pairwise Accuracy (RM1-Honest)
4
UltraFeedback 2% holdout (test)
89.1Pairwise Acc (RM1-Honest)
4
UltraFeedback
71.25Win Rate (RM Evaluator)
3
UltraFeedback Hard Ambiguous (test)
70Ranking Accuracy
3
UltraFeedback Ambiguous (test)
73.4Ranking Accuracy (RA)
3
UltraFeedback Distinct (test)
79Ranking Accuracy (RA)
3
UltraFeedback hard cases
19.03F1-score
2
UltraFeedback
0.17Reward
2
Ultrafeedback
88.8AR Throughput (tok/s)
2