Long-context language understanding on RULER 16K 1.0 (test) benchmark leaderboard · SOTA2 Research

Long-context language understanding on RULER 16K 1.0 (test)

89.28CWE ScoreFull Cache

Evaluation Results

Method	Links
Full Cache2026.05		89.28	90.33	99.6	100	99	98.9	98.9	100	100	100	80.8	57.2	99.72	93.36	—
Full Cache2026.05		82.64	88.07	97.6	95	77.6	88.5	88.65	94.4	96.4	99.6	72	49.8	96.04	86.64	—
CriticalKV w. GRKV2026.05		59.84	79.33	13.6	2.8	0	10.15	9.75	39.2	9.6	2.4	28	31.6	14	23.1	10
CriticalKV2026.05		59.1	78.2	13	2.4	0	10.05	9.45	36.8	9.2	2.4	28.2	31.4	13.24	22.57	0
SnapKV w. GRKV2026.05		47.32	83.33	10.8	2	0.4	9.8	9.7	42.2	4.4	2.4	24.6	30.4	14.32	21.67	10
SnapKV2026.05		45.98	82.8	10.4	1.8	0.4	9.6	9.5	40.8	3.4	2.4	24.2	31.8	12.32	21.18	0
CriticalKV w. D2O2026.05		30.74	71.33	9.8	0.2	0	8.7	9.5	40.4	5.8	2.4	22	29	4.56	18.03	2
SnapKV w. D2O2026.05		22.78	73.13	8.2	0	0	8.7	9.2	43.2	4	2.4	21.6	28.6	2.68	17.27	2
SnapKV w. CaM2026.05		10.32	64.73	10.2	1.4	0	9.05	9.2	5.4	4	2.4	25.4	27.4	1.44	13.15	2
CriticalKV2026.05		9.72	58.73	50.2	5.6	1.6	45.1	46.3	93.6	94.4	2.6	23.8	30.4	68.72	40.83	0
CriticalKV w. GRKV2026.05		8.8	64.93	50	5.8	1.6	45.6	46.65	93.6	95.4	2.6	24	31	69.6	41.51	8
SnapKV2026.05		3.24	59	23.2	4	1.4	17	15.1	78.4	54.2	2.4	21	28	49.72	27.44	0
CriticalKV w. CaM2026.05		2.06	19.8	11.8	0.2	0	9.15	9.2	0	5.6	2.4	26	26.8	0	8.69	0
SnapKV w. GRKV2026.05		1.78	66.4	25.6	4.4	1.4	17.6	15.8	82.4	58.2	2.4	20.2	27.8	54.24	29.09	8
CriticalKV w. D2O2026.05		0.28	53.8	36.4	1	0	36.95	32.45	93.8	80.2	2.4	15	26.2	52.28	33.14	1
SnapKV w. CaM2026.05		0.12	56.2	17.6	2.2	1.6	13.95	13.2	41.4	31.2	2.4	23	28.2	5.44	18.19	3
CriticalKV w. CaM2026.05		0.1	57.87	34.2	2.2	1.2	28.65	28.05	50.6	71	2.6	25.6	25.8	6.48	25.72	1
SnapKV w. D2O2026.05		0.02	57.8	16.4	0.4	0.2	12.7	11.25	79.8	47.6	2.4	14.6	23.8	36.6	23.35	1