Length Extrapolation on RULER (8K-256K Contexts) benchmark leaderboard · SOTA2 Research

Length Extrapolation on RULER (8K-256K Contexts)

92.88Performance @ 8K ContextLlama-3.1-8B-Instruct

Evaluation Results

Method	Links
Llama-3.1-8B-Instruct2026.01		92.88	92.83	89.46	70.79	80.12	72.34	83.47	—
Elastic Attention2026.01		92.82	92	87.8	68.23	78.87	68.51	81.82	0.72
Qwen3-8B2026.01		89.69	85.62	63.23	82.39	65.84	66.71	75.74	—
InfLLM-V22026.01		89.3	80.93	60.98	35.9	32.29	47.27	59.1	—
Qwen3-4B2026.01		87.49	86.82	60.05	70.98	53.19	43.27	66	—
PruLong2026.01		87.2	82.69	61.64	73.05	59.74	58.82	69.9	0.7
DuoAttention2026.01		87.2	77.85	66.99	40.13	57.45	42.92	62.92	0.7
Elastic Attention2026.01		86.62	82.81	64.55	77.41	61.17	61.75	71.74	0.65
Elastic Attention2026.01		86.56	85.38	56.88	69.42	49.48	43.47	63.27	0.67
Elastic Attention2026.01		85.07	85.12	65.08	82.34	64.57	63.41	73.87	0.76
PruLong2026.01		83.54	69.35	56.83	30.88	33.74	21.64	48.82	0.7
Elastic Attention2026.01		83.35	79.24	50.79	67.03	47.83	47.32	61.81	0.66
DuoAttention2026.01		81.03	78.85	56.61	70.1	54.68	56.28	65.94	0.7
Elastic Attention2026.01		80.2	83.42	—	56.3	68.16	56.47	72.85	0.65
InfLLM-V22026.01		78.59	74.4	43.39	44.94	28.57	27.01	51.02	—
PruLong2026.01		77.03	73.99	51.32	60.98	42.7	48.76	58.38	0.7
InfLLM-V22026.01		77.01	68.53	25.93	53.47	34.96	32.95	52.58	—
DuoAttention2026.01		76.82	75.17	50.53	61.42	45.86	45.46	58.3	0.7