AlpacaEval
Benchmarks
TaskDatasetSOTA resultTrendResults
AlpacaEval 2.0
95.87Win Rate
752
AlpacaEval
98.4Win Rate
423
AlpacaEval 2
75.4LC (%)
147
AlpacaEval 2.0 (test)
67.45LC Win Rate (%)
95
AlpacaEval 2
51.9LC Win Rate
89
AlpacaEval (test)
3,213Helpfulness Score
65
AlpacaEval 2
559WR (%)
64
AlpacaEval 2.0 (test)
57.46AlpacaEval (LC win %)
58
AlpacaEval 2.0
49.4Win Rate
58
AlpacaEval 2.0 (test)
30.35LC Win Rate
51
AlpacaEval LC 2
80.9Win Rate
49
AlpacaEval 2.0
648Win Rate
49
AlpacaEval
93.5Win Rate vs Davinci-003
40
AlpacaEval
3,213Win Rate
39
AlpacaEval
72.4Human Agreement
37
AlpacaEval 2
64.2AlpacaEval2 Score
34
AlpacaEval Length-controlled
73.9Score
34
AlpacaEval 2 (val)
82Win Rate
32
AlpacaEval
63.17Score (vs OpenAI)
26
AlpacaEval High-Variance (Top 20%) 2.0
11.6Reward Score
26
AlpacaEval 2.0 (Overall)
11.62Reward
26
AlpacaEval 2.0
61.52LC Win Rate
25
AlpacaEval
98Winrate
25
AlpacaEval
0False Positive Rate
24
AlpacaEval
25.24Win Rate
24