Zero-shot Natural Language Understanding on Standard NLU Suite benchmark leaderboard · SOTA2 Research

Zero-shot Natural Language Understanding on Standard NLU Suite

21.22ARC ChallengeMaxScore

Evaluation Results

Method	Links
MaxScore2025.08		21.22	47.6	63.6	35.57	25.93	69.95	29.9	75.2	67.9	21.6	45.85
MaxScore2025.08		21.11	46.17	64.24	33.38	23.41	67.95	28.9	73.3	64.6	19	44.21
MaxScore2025.08		20.9	44.6	63.73	33.24	23.76	67.63	29.04	73.5	64.41	18.8	43.96
MaxScore2025.08		20.9	45.92	62.39	34	24.96	68.28	29.67	74.3	66.12	19.8	44.63
MaxScore2025.08		20.73	44.49	62.23	32.85	23.27	67.41	28.52	72.5	64	18.4	43.44
MaxScore2025.08		20.73	44.49	62.23	32.85	23.27	67.41	28.52	72.5	64	18.4	43.44
DropLess2025.08		20.14	46.6	61.69	35.11	24.34	68.34	29.19	72.4	67.55	20.2	44.56
DropLess2025.08		20.05	45.24	61.19	33.97	23.6	67.63	27.66	69.7	63.95	17.2	43.02
GShard2025.08		20.05	46.68	63.09	35.14	25.05	69.31	29.19	72.7	67.5	20	44.87
GShard2025.08		19.88	44.49	61.9	31.74	21.54	66.38	28.52	69.4	62.08	16.2	42.11
GShard2025.08		19.8	44.86	62.26	33.05	22.2	67.3	28.46	69.4	63.17	17.6	42.81
GShard2025.08		19.62	44.57	62.28	32.63	21.99	67.19	29.04	69.6	62.77	18.2	42.79
DropLess2025.08		19.62	44.51	62.23	32.36	21.79	67.1	27.46	68.3	62.2	16.8	42.24
DropLess2025.08		19.6	44.69	62.4	32.79	21.65	67.27	27.56	69.5	63.05	17.6	42.61
DropLess2025.08		19.28	44.07	61.16	32.03	21.35	67.14	27.08	67.9	61.55	16	41.76
DropLess2025.08		19.28	44.07	61.16	32.03	21.35	67.14	27.08	67.9	61.55	16	41.76
GShard2025.08		18.86	44.49	61.9	31.74	21.54	66.38	28.52	69.4	62.08	16.2	42.11
GShard2025.08		18.86	45.58	62.16	33.34	23.69	67.74	29.28	70	64.99	19.2	43.59