Long-context language modeling on InfiniteBench (Accuracy and Sparsity) benchmark leaderboard · SOTA2 Research

Long-context language modeling on InfiniteBench (Accuracy and Sparsity)

46.19Code Debug AccuracyFast-dLLM v2

Evaluation Results

Method	Links
Fast-dLLM v22026.06		46.19	—	—	—	—	—	—	—	—	—	—	—	24.29	—	27.12	—	27.12	—	30.32	—	0	57.21
S2O2026.02		24.4	17	91.78	16.6	92.2	27.1	92.25	12	91.5	11	91.06	90.96	34.3	91.54	98.6	93.9	99.8	93.82	37.9	92.11	—	—
Full2026.02		24.1	16.7	0	18.7	0	27.1	0	13.5	0	13.2	0	0	33.1	0	99.3	0	99.3	0	38.3	0	—	—
XAttn2026.02		23.4	18	71.06	16.5	71.32	22.3	71.28	13.5	72.47	8	68.31	73.03	32.9	70.75	99.8	68.92	99.5	68.56	37.1	70.63	—	—
PBS2026.02		22.8	17.9	75.39	19.2	75.78	24.5	75.7	11.5	77.09	12.7	72.92	77.66	32.9	75.13	83.1	74.15	99.2	73.76	36	75.29	—	—
FlexPrefill2026.02		21.8	16.7	85.91	17.8	85.23	24.5	85.34	12.5	84.16	11.3	84.8	83.76	32.8	86.82	100	87.6	98.8	85.1	37.4	85.41	—	—
XAttn2026.02		20	16.4	66.31	1.9	67.01	41.5	66.97	16.5	69.14	3.7	66.69	71.47	30.6	61.89	100	62.29	81	62.12	34.6	65.99	—	—
S2O2026.02		20	17.5	90.85	2	90.89	44.5	89.8	28	86.69	3.8	86.63	87.45	32.3	87.36	100	85.54	99.7	90.33	38.6	88.39	—	—
Full2026.02		18.5	18	0	2.2	0	48.9	0	27.5	0	3.8	0	0	31.7	0	100	0	96.9	0	38.6	0	—	—
PREFILLING-DLLM2026.06		18.02	—	—	—	—	—	—	—	—	—	—	—	24.57	—	95.42	—	70	—	43.62	—	0	53.71
PBS2026.02		17.2	18	77.3	1.9	74.68	47.6	74.73	30.5	76.49	3.8	74.24	78.06	33.7	70.98	100	71.75	100	72.09	39.2	75.04	—	—
FlexPrefill2026.02		17	14.9	82.5	1.9	82.66	45	84.94	16	83.29	3.9	83.28	85.81	34	81.64	99.8	81.81	99	82.37	36.8	83.14	—	—
Fast-dLLM2026.06		8.38	—	—	—	—	—	—	—	—	—	—	—	3.43	—	0.17	—	3.05	—	3.96	—	0	8.73
YaRN2026.06		7.87	—	—	—	—	—	—	—	—	—	—	—	3.43	—	0	—	0	—	2.9	—	0	6.11
Sparse-dLLM2026.06		3.81	—	—	—	—	—	—	—	—	—	—	—	3.43	—	39.32	—	25.59	—	12.24	—	0	1.31
dKV-Cache2026.06		3.05	—	—	—	—	—	—	—	—	—	—	—	4	—	0.51	—	0	—	2.21	—	0	5.68
Sparse-dLLM2026.06		1.52	—	—	—	—	—	—	—	—	—	—	—	7.43	—	8.14	—	2.37	—	4.12	—	0	5.24
PREFILLING-DLLM2026.06		1.52	—	—	—	—	—	—	—	—	—	—	—	26	—	13.22	—	30	—	14.19	—	0	14.41
Vanilla2026.06		0.85	—	—	—	—	—	—	—	—	—	—	—	19.15	—	10.81	—	1.72	—	7.1	—	0	10.09
Fast-dLLM2026.06		0.25	—	—	—	—	—	—	—	—	—	—	—	14.29	—	16.1	—	6.61	—	7.52	—	0	7.86
dKV-Cache2026.06		0.25	—	—	—	—	—	—	—	—	—	—	—	16	—	5.59	—	7.12	—	6.21	—	0	8.3
Vanilla2026.06		0	—	—	—	—	—	—	—	—	—	—	—	34.57	—	1.69	—	2.54	—	12.44	—	0	35.81