Aggregated Benchmarks
Benchmarks
TaskDatasetSOTA resultTrendResults
Aggregated Benchmarks GSM8K MBPP HumanEval ARC-E ARC-C
61.16Average Score
26
Aggregated Benchmarks
49.76Overall Average
22
Aggregated Benchmarks HellaSwag, MMLU, Arc-C, MATH-500
1.014Average Score
22
Aggregated Benchmarks (DABench, DSBench, TableBench, DABStep-R)
73.68Overall Score
18
Aggregated Benchmarks (AIME, ARC, GSM8K, HE, MMLU, IT, RU, BFCL)
2.4Overall Average Rank
16
Aggregated Benchmarks Average
61.2Average F1
14
Aggregated Benchmarks
71Average Score
13
Aggregated Benchmarks
42.2Average Score (3-item avg)
12
Aggregated Benchmarks Macro
74.74Average Score (excl. MM-RB, VL-RB)
12
Aggregated Benchmarks
0.745Average Score
10
Aggregated Benchmarks
1.3Average Rank
8
Aggregated Benchmarks HELMET, LongBench, RULER, Reasoning
41.7HELMET Score
8
Aggregated Benchmarks STEM Code IF General
61.7Average Score
7
Aggregated Benchmarks Math500, GPQA, HumanEval, MBPP, AE2 LC
40.7Average Score
5