MMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
MMLU Med
90.6Accuracy
34
MMLU
63.73MMLU Score
33
MMLU-Redux (test)
69.8Knowledge Accuracy
33
MMLU-Pro
37.42MMLU-Pro Score
33
MMLU-Redux
85.75Accuracy
33
MMLU
75.05Accuracy
33
MMLU
68.6Humanities Avg
33
Bangla MMLU 1.0 (test)
35Accuracy
33
MMLU Pro (test)
92.8Physics Score
32
MMLU-Pro standard (test)
80.8Accuracy
32
MMLU Spanish (test)
86Accuracy
32
MMLU English (test)
89Accuracy
32
MMLU-Pro
80.3Pass@1
32
MMLU algebra
97Pass@1
31
MMLU
78.74Accuracy (5-shot)
31
MMLU
84.9Accuracy
31
MMLU-Pro
87.08AUROC
31
MMLU
72.48pass@1
31
M-MMLU
77.83Overall Accuracy
31
MMLU
82.33RA
31
MMLU formal_logic (test)
66.6Accuracy (MMLU FLR Test)
30
MMLU
86Task Success Rate (TSR)
30
MMLU
37.28Delta Accuracy
30
MMLU
53Accuracy
30
MMLU
85.67Accuracy
30