Language Modeling on LM-Harness (Standard Suite Metrics) benchmark leaderboard · SOTA2 Research

Language Modeling on LM-Harness (Standard Suite Metrics)

32.85ARC-C AccuracyDeltaNet

Evaluation Results

Method	Links
DeltaNet2025.11		32.85	58.16	42.51	81	61.13	73.78	43.9	61.72	11.8	46.08	51.29
Gated DeltaNet2025.11		32.59	60.02	62.75	82	62.8	74.32	80.6	62.35	8.26	44.28	57
Gated KalmaNet2025.11		32.51	59.89	61.68	85	63.84	74.81	83.2	64.17	12.89	50.95	58.89
Transformer2025.11		32.25	56.1	64.28	80	60.96	73.56	79.5	61.72	58.53	72.28	63.92
Mamba22025.11		32.24	59.64	58.72	82	62.23	73.78	79.8	62.19	7.71	41.13	55.94
Mamba22025.11		28.07	46.63	60.21	70	44.57	67.57	65.5	54.3	1.45	15.75	45.4
Gated Linear Attention2025.11		27.82	50.8	52.57	78	48.83	70.13	69.6	54.54	2.81	20.43	47.55
DeltaNet2025.11		27.56	46.25	59.97	71	43.18	67.74	65.9	55.41	3.09	20.61	46.07
Gated DeltaNet2025.11		27.05	47.98	59.54	74	44.27	67.36	66.2	53.83	2.18	17.82	46.02
Transformer2025.11		26.62	46.42	59.94	77	44.01	67.14	68.3	54.06	8.35	45.18	49.7
Gated Linear Attention2025.11		25.94	42	58.84	70	36.34	63.6	58.2	51.85	1.45	10.53	41.88
Gated KalmaNet2025.11		25.43	46.55	60.73	74	44.59	68.88	67.6	52.41	6.17	21.87	46.82
Gated DeltaNet2025.11		25.17	41.96	58.23	72	36.96	64.69	63.6	51.7	1.91	11.88	42.81
DeltaNet2025.11		25.09	41.92	61.13	65	37.2	64.47	64	49.49	2.81	14.31	42.54
Gated KalmaNet2025.11		24.57	43.22	56.94	71	37.22	64.47	62.8	50.83	1.45	14.04	42.65
Transformer2025.11		24.4	42.26	59.88	70	36.19	64.15	61.5	51.7	5.17	35.64	45.09
Gated Linear Attention2025.11		24.06	40.28	56.57	71	32.7	62.24	57.8	50.67	1	9.18	40.55