MMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
MMLU-Pro
32.1Math Accuracy
6
MMLU-Pro
40.4Math Score
6
MMLU-Pro
18.5Math Score (MMLU-Pro)
6
MMLU 7-shot (test)
70.13Humanities Accuracy
6
MMLU 0-shot (test)
70.21Humanities Accuracy (0-shot)
6
MMLU Redux
91.7Knowledge Accuracy (MMLU Redux)
6
MMLU
58.3Accuracy
6
MMLU
88.25Task Success Rate (TSR)
6
MMLU HS-Math
94.81Accuracy (Raw consensus)
6
MMLU Pro
64.46Accuracy
6
MMLU-Pro
21Accuracy
6
MMLU out-of-domain
48.41Logic Accuracy
6
MMLU Elementary
96Accuracy
6
MMLU High School
88.1Accuracy
6
MMLU-Pro
0.88Spearman Rank Correlation
6
MMLU filtered subset (16 subjects)
2.01ASR@30
6
MMLU (Continual)
32.03Accuracy
6
MMLU vanilla
0P-value
6
MMLU
82.7STEM Accuracy
6
MMLU
49.39Pass@1 Accuracy
6
MMLU (val)
6,402Answered Count
6
MMLU
90.13Selection Accuracy
6
MMLU
68MMLU Accuracy
6
MMLU
64.21Score
6
MMLU-Pro
56pass@1
6