Long-context language understanding on RULER 64k context length benchmark leaderboard · SOTA2 Research

Long-context language understanding on RULER 64k context length

0FWE (Error)Full Attention

Evaluation Results

Method	Links
Full Attention2026.02		0	99.4	98.4	96.8	93.7	19.8	70.4	47.6	65.8	—	—	—	—	—	—	—
LycheeDecode2026.02		0	99.6	73.2	90.3	81.7	19.9	63.4	44.4	59	—	—	—	—	—	—	—
E2LLM-C2024.09		6.33	—	—	—	—	0.6	—	—	—	—	29.5	2.63	4.9	—	—	—
E2LLM-R2024.09		10.67	—	—	—	—	4.4	—	—	—	—	1	50.13	2.6	—	—	—
LongLLMLingua2024.09		13.33	—	—	—	—	0.2	—	—	—	—	15	23.25	0.3	—	—	—
LLoCO2024.09		15.67	—	—	—	—	0	—	—	—	—	9	0.31	2.4	—	—	—
MiniCache2025.03		18.4	35.42	—	1.56	1.82	1.46	20.83	30.21	13.78	—	—	—	—	12.5	13.54	2.08
E2LLM-C+R2024.09		32.67	—	—	—	—	6.2	—	—	—	—	34.5	9.34	0.2	—	—	—
Training–Inference Consistent Segmented Execution2026.05		34.17	—	—	—	—	—	—	—	—	—	—	—	2	—	—	—
MiniCache2025.03		58.33	89.58	—	21.35	14.06	49.38	61.46	35.42	45.04	—	—	—	—	66.67	43.75	10.42
Single SVD2025.03		61.11	25	—	44.79	28.91	3.54	47.92	36.46	45.71	—	—	—	—	51.04	61.46	96.88
PyramidKV2025.03		68.06	100	—	98.44	100	95.42	83.33	57.29	89.94	—	—	—	—	100	100	96.88
SnapKV2025.03		68.75	100	—	97.66	100	95	83.33	58.33	89.68	—	—	—	—	100	98.96	94.79
xKV2025.03		78.47	100	—	96.62	96.09	86.67	78.13	56.25	88.5	—	—	—	—	96.88	97.92	97.92
Single SVD2025.03		80.21	94.79	—	66.15	82.81	46.04	48.96	50	71.79	—	—	—	—	66.67	83.33	98.96
KIVI-22025.03		84.38	100	—	89.58	91.41	81.46	80.21	55.21	86.87	—	—	—	—	96.88	98.96	90.63
Full Attn2025.03		85.42	100	—	97.66	98.96	97.29	83.33	59.38	91.89	—	—	—	—	100	98.96	97.92
xKV-42025.03		85.76	100	—	91.93	100	95	71.88	60.42	90.19	—	—	—	—	97.92	100	98.96
PyramidKV2025.03		86.11	100	—	97.14	99.74	99.58	83.33	64.58	91.17	—	—	—	—	100	100	81.25
KIVI-22025.03		89.58	100	—	92.97	98.18	97.71	78.13	64.58	90.45	—	—	—	—	98.96	95.83	88.54
SnapKV2025.03		90.28	100	—	95.57	99.74	99.79	81.25	65.63	91.66	—	—	—	—	100	100	84.38
Full Attn2025.03		91.32	100	—	98.96	100	99.58	80.21	64.58	93.36	—	—	—	—	100	100	98.96
CLAA2026.02		—	—	87.72	86.67	—	—	—	—	82.01	89.85	63.8	—	—	—	—	—
FastKV2026.02		—	—	87.16	86.67	—	—	—	—	81.28	87.69	63.6	—	—	—	—	—
FullKV2026.02		—	—	86.88	86.67	—	—	—	—	83.78	98.27	63.3	—	—	—	—	—
GemFilter2026.02		—	—	58.2	88.33	—	—	—	—	73.12	82.23	63.7	—	—	—	—	—