Aggregate
Benchmarks
TaskDatasetSOTA resultTrendResults
Aggregate (AIME, AMC, MATH, Minerva, Olympiad)
48.6Average Score
10
Aggregate (AMOS, BraTS, CoW, HVSMR, PanDG, Prostate)
1.5Average Rank
10
Aggregate (MATH-500, AIME-2024, GSM8K, HumanEval+, MBPP+)
83.17Average Score
10
Aggregate MATH, OlympiadBench, GSM8K, BBH, MMLU-CF, LongBench, HotpotQA, MuSiQue
63.88Average Score
10
Aggregate (LAMBADA, HellaSwag, PIQA, ARC, WinoGrande) (various)
51.9Avg Accuracy
10
Aggregate HumanEval, MBPP, GSM8K, Minerva Math, MMLU, ARC, HellaSwag, PIQA, WinoGrande
72.36Average Accuracy
9
Aggregate (MMSI, SAT, MMVP, CV-Bench 3D, BLINK, VStar, CV-Bench 2D)
70.53Average Score
9
Aggregate
68.73Average Score
9
Aggregate
90.7Micro-F1
9
Aggregate All Datasets
1.6Rank
9
Aggregate (NuScenes, AV2, Waymo, ONCE, NuPlan)
1.8Average Rank
9
Aggregate EMMA, VisuLogic, Zebra-CoT
37.5Total Score
9
Aggregate (cat_adoption, mgus, nafld1, nwtco, std, synthetic-breakpoint, synthetic-tve, tumor, wa_churn) (outer repeated cross-validation)
69.11Harrell’s C
8
Aggregate All Datasets
2.4Total Rank
8
Aggregate (AIME, MATH-500, GPQA-D, LCB)
52Average Score
8
Aggregate (HealthQA, ARC-C, PopQA, Squad1, ASQA)
62.8Average Score
8
Aggregate c-MET, BRD4, ACAA1
1.56Average Rank
8
Aggregate 16 Single-Session sequences
78.2Average AUC
8
Aggregate RewardBench PPE RMB RM-Bench JudgeBench
82.8Average Score
8
Aggregate (BrowseComp, BrowseComp-zh, GAIA, HLE)
69.3Overall Score
8
Aggregate Audio-Visual & Video Benchmarks
57.6Avg Audio-Visual Score
8
Aggregate 8 tabular datasets
2.21Average Rank
8
Aggregate All Tasks
59.5Accuracy
7
Aggregate ISM330DHCX, MPU6050, LSM6DSOX
0.001Average Error (degrees)
7
Aggregate Malaria Camelyon17-WILDS
31.7APU
7