Multi-task Language Understanding
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
74Accuracy
5
Feb 26, 2026
78Accuracy
5
Feb 26, 2026
79Accuracy
5
Feb 26, 2026
49.1Anachronisms
5
Feb 26, 2026
83.01MMLU Accuracy (Vanilla)
4
Jun 23, 2026
33.52Throughput (TPS)
4
Jun 4, 2026
45.75Throughput (TPS)
4
Jun 4, 2026
38.01Tokens Per Second (TPS)
4
Jun 4, 2026
35.86TPS
4
Jun 4, 2026
69.5MMLU Accuracy
4
May 12, 2026
60.8Normalized Log Accuracy
4
Mar 18, 2026
71.82Accuracy
4
Feb 26, 2026
64.47MMLU Pass@1 Accuracy
4
Feb 26, 2026
86.4Accuracy
4
Feb 26, 2026
94.4Accuracy
3
May 20, 2026
14.56Throughput (Tokens/s)
3
Apr 21, 2026
17.3TPS
3
May 12, 2026
36.7Accuracy
2
Jun 4, 2026
66.1E*S Combined Score
2
Jun 4, 2026
61.2E*S
2
Jun 4, 2026
56MMLU Score
2
May 15, 2026
54.56Throughput
2
Feb 26, 2026
72.4Accuracy
2
Feb 26, 2026
—
—Primary metric
0
Apr 23, 2026