Multitask Language Understanding on MMLU (Accuracy and Resource Usage) benchmark leaderboard · SOTA2 Research

Multitask Language Understanding on MMLU (Accuracy and Resource Usage)

69.62AccuracyVanilla

Evaluation Results

Method	Links
Vanilla2026.04		69.62	14.82	2,883	6.9
CoT2026.04		66.5	1.77	649	0.2
Vanilla + H2O2026.04		65.92	27.11	736	1.8
CoT2026.04		65.82	2.39	736	0.3
LThinker_tho2026.04		63.39	14.88	882	1.8
LThinker_tok2026.04		61.05	15.85	1,538	3.3
LThinker_tho2026.04		60.47	13.09	944	1.9
Vanilla2026.04		59.98	20.61	3,417	10
Vanilla + H2O2026.04		59.69	29.02	1,024	3.2
AnLLM2026.04		58.62	16.53	589	1.2
LThinker_tok2026.04		57.35	13.8	489	3.5
AnLLM2026.04		54.63	14.13	875	2
Distill-R12026.04		53.46	2.97	582	0.8
Distill-R12026.04		51.7	14.31	2,483	7.5
Vanilla + SepLLM2026.04		25.12	50.11	1,024	7.5
Vanilla + SepLLM2026.04		10.81	53.45	1,024	9