Multi-task Language Understanding on MMLU (Calibration Metrics) benchmark leaderboard · SOTA2 Research

Multi-task Language Understanding on MMLU (Calibration Metrics)

0.087Error Rate (ER)Seq. Post.

Evaluation Results

Method	Links
Seq. Post.2026.05		0.087	0.179	62.2	0.545	12.8	10.8	0.416	21.5	9.5
Is True2026.05		0.087	0.086	52.3	0.653	8.7	10.2	0.653	17.4	8.8
Verbalized2026.05		0.095	0.037	66.9	0.455	8.5	5.8	0.461	18	9.6
Verbalized2026.05		0.126	0.094	64.6	1.054	11.4	7.3	1.059	24	12.7
Seq. Post.2026.05		0.145	0.131	72.6	1.003	13.5	8.5	0.979	28	14.8
Is True2026.05		0.145	0.135	54.7	1.346	14.3	13.8	1.347	28.8	14.6
Seq. Post.2026.05		0.181	0.083	81.6	0.422	12.7	5.9	0.455	30.8	18.2
Is True2026.05		0.181	0.152	52.7	1.07	17.2	19.1	1.074	35.3	18.2
Verbalized2026.05		0.187	0.104	62.4	0.599	15.6	12.3	0.616	34.3	18.8
Seq. Post.2026.05		0.234	0.046	84	0.409	13.1	7.2	0.474	36.5	23.6
Is True2026.05		0.234	0.175	52.6	0.793	21.1	21.5	0.804	44.5	23.6
Verbalized2026.05		0.234	0.185	61.1	1.724	20.7	18.5	1.711	44.1	23.8
Seq. Post.2026.05		0.264	0.151	80.9	0.591	18.2	10.3	0.64	44.6	26.6
Is True2026.05		0.264	0.193	50.2	0.671	23.9	54.4	0.669	50.3	26.6
Seq. Post.2026.05		0.265	0.03	82.4	0.442	14.6	9.3	0.516	41.1	26.7
Is True2026.05		0.265	0.092	49.4	0.623	20.7	27.2	0.654	47.2	26.7
Verbalized2026.05		0.273	0.217	60.4	2.049	23.9	22.2	2.053	51.2	27.6
Verbalized2026.05		0.28	0.202	57	1.193	24	23	1.157	52	28.6
Verbalized2026.05		0.327	0.234	56.5	1.148	27.2	29.9	1.049	59.9	33.8
Seq. Post.2026.05		0.372	0.316	54.1	0.889	33.3	45.8	0.821	70.5	37.5
Is True2026.05		0.372	0.141	50.5	0.771	26.4	37.8	0.832	63.6	37.5