MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU Pro
64.3Level Score
16
MMLU-Pro (full)
61Overall Accuracy (MMLU-Pro QA)
16
MMLU 5-shot (test)
67.2Utility Score
16
MMLU CoT
87.2Accuracy
16
MMLU
1.2Average Rank
16
MMLU-Pro
88.6Accuracy
16
MMLU Camel
0.715AUROC
16
MMLU
59.25AUROC
16
MMLU AutoGen (test)
0.732AUROC
16
MMLU
69.62Accuracy
16
MMLU
77.37Accuracy
16
MMLU-IT
81.5Accuracy
16
MMLU Pro
10.12Throughput
16
MMLU random topology
15.5ASR (k=1)
16
MMLU (test)
100Accuracy
16
Global MMLU
36.1MMLU
16
MMLU
79.7Accuracy
16
MMLU (test)
75.46Accuracy
16
MMLU
56.6MMLU
16
MMLU phy
99.3ASR
16
MMLU bio
98.4Attack Success Rate (ASR)
16
Multilingual MMLU internal translated version
85.5Accuracy
16
MMLU-Pro
57.2Accuracy
16
MMLU
82.4EM
16
MMLU (Massive Multitask Language Understanding) 1.0 (test)
88.7Accuracy (Clinical knowledge)
16