MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Mmlu Multi-Answer
61.45Overall Score
4
MMLU-Pro
85.3Pass@1
4
MMLU-K
76.37Accuracy
4
MMLU Pro
42.4Normalized Log Accuracy
4
MMLU Full Text
69.8Normalized Log Accuracy
4
MMLU
69.7Normalized Log Accuracy
4
MMLU-Pro
18Exact Match Accuracy
4
MMLU redux
85.11Accuracy
4
MMLU-Pro In-Distribution (test)
83.57CPT (90%)
4
MMLU-Redux In-Distribution (test)
75.06CPT (90%)
4
MMLU In-Distribution (test)
76.3CPT (90%)
4
MMLU-Redux OOD
63.09CPT (85%)
4
MMLU OOD
63.55CPT (85%) Score
4
MMLU-Pro
91.42CPT (95%)
4
MMLU-Redux
86.79CPT (95%)
4
MMLU
88.15CPT Score (95%)
4
MMLU-Pro
82.65CPT (90%)
4
MMLU-Redux
74.61CPT (90%)
4
MMLU
74.53CPT (90%)
4
MMLU-Pro (ID)
68.24CPT (80%)
4
MMLU-Redux (ID)
52.93CPT (80%)
4
MMLU (ID)
52.61CPT (80%)
4
MMLU-Pro (ID queries)
75.91CPT Score (85%)
4
MMLU-Redux (ID queries)
63.77CPT (85%)
4
MMLU (ID queries)
64.46CPT (85%) Score
4