MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU STEM (test)
72.4Accuracy
2
MMLU
1.3MMLU General Knowledge Delta (pp)
1
MMLU Biology
0Accuracy
1
MMLU Chemistry
0Calibration Rate
1
MMLU Physics
54Correctness Rate (CR)
1
MMLU CS (LIFO)
0Accuracy (CR)
1
MMLU Math
49Correctness Rate (CR)
1
MMLU-Pro
77.54AS
1
MMLU-Hard abstract_algebra 100 items
41.7Baseline Accuracy
1
MMLU (test)
90.93Accuracy
1
MMLU
83.7Accuracy
1
MMLU-Pro
32.13Before Score
1
MMLU Medical
—Primary metric
0
MMLU
—Primary metric
0
MMLU
—Primary metric
0
MMLU (test)
—Accuracy (Learned)
0
MMLU
—Learned Knowledge
0
Global MMLU-Lite 21 (test)
—Score
0