Long-context language modeling on LongBench-E 1.0 (test)

49.92S-Doc QA Perf.Elastic Attention

Evaluation results

Method
Elastic Attention2026.01	49.92	0.64	48.92	0.63	30.17	0.73	67.99	0.74	54	0.64	60.7	0.79	53.35	0.69
Elastic Attention2026.01	49.4	0.71	52.94	0.69	30.3	0.8	68.55	0.79	49.66	0.72	56.49	0.87	52.71	0.77
XAttention2026.01	48.82	—	51.73	—	30.26	—	68.57	—	40.83	—	56.53	—	51	—
Llama-3.1-8B-Instruct2026.01	48.75	—	51.85	—	30.26	—	68.16	—	56	—	55.81	—	53.28	—
Elastic Attention2026.01	48.3	—	46.4	—	30.27	—	67.83	—	41.37	—	60.81	—	50.71	—
MoBA2026.01	46.63	—	43.91	—	30.72	—	66.78	—	36	—	64.48	—	49.69	—
Elastic Attention2026.01	46.15	0.64	46.54	0.65	28.19	0.72	67.52	0.71	48.07	0.65	62.95	0.78	51.51	0.69
Qwen3-8B2026.01	45.57	—	51.59	—	28.34	—	66.64	—	50.16	—	61.2	—	52.16	—
MoBA2026.01	44.73	—	43.28	—	30.33	—	64.22	—	48.95	—	62.55	—	50.47	—
Elastic Attention2026.01	44.4	0.68	39.42	0.71	28.49	0.82	65.26	0.76	44.35	0.74	54.29	0.87	47.59	0.76
Elastic Attention2026.01	44.01	0.75	49.99	0.76	28.3	0.83	66.23	0.8	50.95	0.77	60.57	0.86	51.66	0.8
Qwen3-4B2026.01	43.69	—	38.48	—	28.46	—	66.21	—	49.59	—	54.38	—	48.45	—
XAttention2026.01	43.48	—	48.99	—	28.39	—	66.26	—	42.86	—	60.62	—	50.13	—
NSA2026.01	42.33	—	40.37	—	29.93	—	66.65	—	15.17	—	57.65	—	44.03	—
Elastic Attention2026.01	42.2	0.66	38.86	0.68	28.5	0.76	65.73	0.73	48.43	0.71	54.34	0.82	48.08	0.73
Elastic Attention2026.01	41.92	—	38.67	—	28.45	—	65.25	—	49.25	—	55.07	—	48.14	—
XAttention2026.01	41.58	—	38.85	—	28.76	—	65.45	—	39.01	—	54.52	—	46.44	—
NSA2026.01	40.63	—	37.45	—	27.97	—	67.14	—	29	—	62.19	—	46.12	—
Elastic Attention2026.01	39.43	—	41.38	—	28.27	—	66.19	—	46.73	—	61.85	—	49.25	—
NSA2026.01	39.13	—	34.81	—	27.38	—	63.44	—	23.8	—	58.1	—	43.02	—
MoBA2026.01	38.16	—	34.3	—	29.46	—	64.57	—	37.42	—	54.62	—	45.09	—
Quest2025.12	30.76	—	19.97	—	27.04	—	78.84	—	—	—	50.26	—	-0.39	—
ShadowKV2025.12	30.73	—	17.88	—	30.71	—	78.95	—	—	—	49.15	—	-0.27	—
Flat2025.12	29.94	—	24.01	—	28.92	—	78.35	—	—	—	50.26	—	0.54	—
ShadowKV2025.12	29.9	—	30.32	—	27.71	—	83.61	—	—	—	69.31	—	-0.98	—
CTkvr2025.12	29.68	—	21.2	—	28.98	—	78.12	—	—	—	50.29	—	-0.11	—
CTkvr2025.12	29.55	—	21.25	—	29.32	—	78.12	—	—	—	50.41	—	-0.03	—
FullKV2025.12	29.13	—	21.82	—	29.33	—	79.04	—	—	—	49.5	—	41.76	—
Flat2025.12	29.02	—	37.99	—	26.29	—	83.61	—	—	—	69.12	—	0.05	—
CTkvr2025.12	28.77	—	38.08	—	26.24	—	83.61	—	—	—	69.35	—	0.06	—
CTkvr2025.12	28.72	—	38.16	—	26.15	—	83.61	—	—	—	69.72	—	0.12	—
Quest2025.12	28.19	—	37.39	—	25.97	—	83.34	—	—	—	68.46	—	-0.48	—
FullKV2025.12	28.04	—	38.54	—	25.86	—	83.5	—	—	—	69.81	—	49.15	—
MagicPIG2025.12	26.67	—	38.21	—	24.74	—	83.66	—	—	—	69.14	—	-0.66	—
MagicPIG2025.12	18.17	—	9.59	—	26.79	—	78.08	—	—	—	48.82	—	-5.47	—
SnapKV2025.12	8.83	—	10.13	—	11.29	—	52.38	—	—	—	25.16	—	-27.59	—
SnapKV2025.12	5.42	—	6.25	—	12.35	—	40.12	—	—	—	27.06	—	-23.52	—