Average
Benchmarks
TaskDatasetSOTA resultTrendResults
Average Zero-shot Reasoning
61.76Accuracy
11
Average (HotpotQA, 2WikiMultiHopQA, MuSiQue, Medical)
56.57Containment Acc.
11
Average Code, Law, Med
133.31Throughput (tokens/s)
11
Average MMLU-Pro, MATH, TACO
59Average Accuracy (AA)
11
Average across OOD datasets
88.47AUC
11
Average Narrow-band Notch
90.47AUC
11
Average Aggregate of 6 Attacks
0Attack Success Rate (ASR)
11
Average All Benchmarks 10% data
0.372MSE
11
Average nuScenes, Waymo, KITTI
100mAP
11
Average SST2, AGNEWS, GSM8K
26.47HS Score
11
Average 4x scale (test)
33.06PSNR
11
Average 2x scale (test)
40.16PSNR
11
Average Covariate Shift
39.9Metric A
11
Average (Five Tasks)
76.8Task Success Rate (S)
11
Average
51.86R@1
11
Average GPQA, LSAT, MMLU-1000
0.38Pass@1
11
Average NER, RE, ET
74.18Precision
10
Average IN-C, IN-R, DN-126
62.5Average Accuracy
10
Average
80Score
10
Average Olympiad, MATH, AMC, AIME, GPQA
69.2Average Accuracy
10
Average
80.46Accuracy
10
Average t=1
64.73Accuracy
10
Average t=2
42.46Accuracy
10
Average
0.72Kendall Correlation
10
Average MIDOG, X-Ray, Derma
86.01AUROC
9