Average
Benchmarks
TaskDatasetSOTA resultTrendResults
Average 11 datasets
87.55Base Accuracy
95
Average 7 Commonsense Reasoning Tasks
72.04Avg Accuracy
72
Average across 15 datasets (test)
81.1Top-1 Accuracy
60
Average Deepglobe, ISIC, Chest X-Ray, FSS-1000
76.9mIoU
57
Average OpenImage-O, Texture, iNaturalist, ImageNet-O
96.23AUROC
54
Average (MuSiQue, HotpotQA, 2WikiMultiHopQA, LongSeal)
46.49Average QA Score
50
Average Cross-domain
0.787Mean AUROC
48
Average 11 datasets (test)
87.41Average Accuracy (Few-shot)
47
Average Overall
64.3mIoU
46
Average of 5 datasets
78.9Average Score
46
Average Bach Counting Blues v1 (test)
32.66SNR
46
Average GSM8k-Aug, GSM-Hard, SVAMP, MultiArith
79.3Accuracy
42
Average All shifts (test)
90.99AUC
40
Average
89.7Accuracy
36
Average (FTP-75 and PDMHC)
0.412MAE
35
Average across datasets (combined)
37.6PSNR
35
Average base-to-novel
85.55Base Score
33
Average fine-grained
81.49cACC (All)
32
Average
0.47Weighted Kendall's Tau (w)
32
Average (ACDC, M&Ms, RandBias, RandGhosting, RandMotion, RandSpike) (out-of-domain)
81.25Dice Score
32
Average (OBQA, ARC, Riddle, PQA)
68.31Average Accuracy
31
Average
26.47FPR@95
31
Average Across all benchmarks
6.9Speedup
28
Average GSM8k, GSM-sym, MATH, NuminaMath, AIME
71.41Accuracy
28
Average (MATH, OlympicBench, MinervaMath, AMC2023, GPQA) (test)
59.83Pass@1
27