MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU
28.36Chemistry (Pro)
8
MMLU Basque
79.63Accuracy (MMLU Basque)
8
MMLU EL
44.06MMLU EL (General Knowledge) Accuracy
8
MMLU
1.49Speedup
8
MMLU (test)
84.97gpt4o-mini Score
8
MMLU v1 (test)
79.48Accuracy
8
MMLU
47Comparable SC Samples
8
MMLU High School History 1.0 (test)
92.83Accuracy
8
MMLU College Medicine 1.0 (test)
86.13Accuracy
8
MMLU
34Macro Similarity Type
8
MMLU subsampled 2500 s
73.9Accuracy
8
MMLU-ST
89.6Accuracy
8
MMLU-Pro
0.83AUROC
8
MMLU
53.43MMLU Score
8
MMLU-Pro
43.3Accuracy
8
MMLU
44.4Sycophancy Rate
8
MMLU Code Switched P, I - (EN), Q(X)
99.87Language Consistency
8
MMLU 0-shot (test)
64.75Pass@1 Accuracy
8
MMLU
72.7MMLU Accuracy (Before Attack)
8
MMLU
2.8Mu Bias (STEM)
8
MMLU (dev)
55.6Accuracy
8
MMLU
11,691.83Avg Input Context Length (tokens)
8
MMLU
61.39Accuracy
7
MMLU MINCE Calibration n=1500
83.2Accuracy
7
TR-MMLU
83.5Token Purity
7