List your company

ResearchFormerly WizWand

UltraFeedback

Benchmarks

TaskDatasetSOTA resultTrendResults

UltraFeedback clean (held-out)

0.362Kendall’s Tau

60

UltraFeedback In-domain v1 (test)

81Win Rate

46

UltraFeedback

8.1MT-Bench Score

42

UltraFeedback

30LC

42

Reward Modeling

UltraFeedback (test)

0.145MAE

38

Controllable Generation

Code-UltraFeedback

90.7Diversity

36

Generative Performance

Ultrafeedback 61.1k (test)

69.8Win Rate

30

Discriminative Performance

Ultrafeedback 61.1k (test)

73.05Accuracy

30

Ultrafeedback

86Win Rate

24

Response Generation

UltraFeedback (val)

88.1BERTScore

24

Preference Alignment

UltraFeedback

83Win Rate

24

UltraFeedback

68.75Accuracy

23

Multi-turn Conversation Evaluation

UltraFeedback

6.1MT-Bench Score

20

Sequential Preference Optimization

UltraFeedback

99.71Harmless Rate

20

UltraFeedback (test)

63.72Honesty Score

20

General Evaluation

UltraFeedback Aggregate

59.59Overall Average Score

18

Best-of-N Reward Evaluation

UltraFeedback core250

24.323Reward Score

18

Reward Modeling

UltraFeedback core250 (held-out evaluation)

3.543Delta (Δ)

18

UltraFeedback (test)

21AlpacaEval 2 Win Rate (WR)

18

Reward Model Transfer

UltraFeedback (UF)

7.93AOG

16

Instruction Following

UltraFeedback (core250)

12.568Delta Preference Score (bo64)

15

Reward Modeling

UltraFeedback

0.794R2 (Top-50)

14

Pairwise Judge Comparison

UltraFeedback core250

161Win Count (W)

14

Response Preference Evaluation

UltraFeedback (test)

83.42Win Rate

14

Preference Evaluation

UltraFeedback core250 (test)

80Win Rate

12