Aggregate
Benchmarks
TaskDatasetSOTA resultTrendResults
Aggregate (GPQA-D, GSM8K, HumanEval, MATH-500, MBPP, MMLU-Pro)
75.9Average Accuracy
66
Aggregate MVBench, LongVideo Bench, MLVU, VideoMME
100Average Accuracy
63
Aggregate All 11 Datasets
55.7Mean PRR
44
Aggregate of 8 tasks zero-shot
76.31Accuracy (Zero-Shot Aggregate)
35
Aggregate 9 benchmarks
64.9Average Score
34
Aggregate
100Relative Performance
33
Aggregate
1Avg Rank
33
Aggregate AMC23, AIME24, MATH-500, Minerva, Olympiad
68.2Intelligence Per Token (IPT)
30
Aggregate 10-Benchmark Suite
79.9Average Score
29
Aggregate (MATH, GSM8K, H_Eval, MMLU, CEVAL, CMMLU, BoolQ, CSQA)
81.41Aggregate Accuracy
26
Aggregate
100Relative Performance
25
Aggregate BUS-BRA, GIST514-DB, BreastMNIST, Breast
1.5Average Rank
24
Aggregate All Unseen (test)
98.06mAP
24
Aggregate (test)
4.97Comprehensiveness
24
Aggregate All Datasets
56.3Mean PRR
22
Aggregate (Open-WikiTable, 2WikiMQA, InfoSeek, Dyn-VQA, TabFact, WebQA)
55.93Average Score
22
Aggregate ARC-C, MMLU, HellaSwag, TruthfulQA (test)
159.215Total Score
22
Aggregate
69.86Average Score
21
Aggregate (AIME25, AIME24, MATH, GSM8K, HumanEval+, MBPP+, MedQA, GPQA-Diamond)
77.6Average Accuracy
21
Aggregate Mean over Alpaca, CodeAlpaca, HumanEval, LiveCodeBench, Math500, MBPP, MT-Bench
2.87Mean Speedup
21
Aggregate
80Macro Score
21
Aggregate 5 Unseen Languages
75.1Average Score
20
Aggregate 8 Seen Languages
69.5Average Score
20
Aggregate (GSM8K, BFCL, Spider, HumanEval)
79.4Average Accuracy
20
Aggregate BFCL and Meta-Tool
86.6Accuracy
20