Long-context language modeling on Ruler llama3-8B-Instruct (test) benchmark leaderboard · SOTA2 Research

Long-context language modeling on Ruler llama3-8B-Instruct (test)

100S-NIAH-1FullKV

Evaluation Results

Method	Links
FullKV2026.03		100	100	100	99.2	91.8	95.8	99.75	97.5	97.82	82.93	98.32	96.65
H2O2026.03		100	90	14	78	47	12.6	74.7	45	90.04	79.33	97.68	66.21
PyramidKV2026.03		100	100	40.8	99	72.2	17.4	99	96.3	83.25	67.87	98.2	79.46
SnapKV2026.03		100	98.4	61.4	99	69	19.8	98.8	94.45	90.34	73.13	97.56	81.99
DapQ2026.03		100	99	96	99.2	67.2	24.4	99.05	95.45	90.68	75	97.64	85.78
PyramidKV2026.03		100	98.2	6.2	98.6	47.8	2	97	90.8	56.14	61.53	97.08	68.67
SnapKV2026.03		100	96.8	15.4	98.4	44.8	3.4	96.45	87.9	65.42	64.27	96.48	69.94
DapQ2026.03		100	98.4	85.8	99.2	41.8	6.2	97.4	92.5	65.74	69.33	96.76	77.56
PyramidKV2026.03		100	95.6	0	97.4	35	0.2	91.8	73.5	23.56	52.8	92.96	60.26
SnapKV2026.03		100	95.6	—	96.8	30.4	0.4	91.1	71.65	30.82	53.73	94.48	60.58
DapQ2026.03		100	97.8	59.6	98.6	29.8	1	91.95	82.9	27.16	61.27	95.2	67.75
PyramidKV2026.03		100	94.8	0	89.6	29.4	0	73.5	35.15	9.42	39.8	75.8	49.77
SnapKV2026.03		100	95	0	90.2	26	0	76.5	36.3	13.66	45.2	91.56	52.22
DapQ2026.03		100	97.6	23	97.8	19.8	0	79.7	55	12.8	51.87	88.04	56.87
DapQ2026.03		99.6	97.6	1.4	94.4	21.4	0	28.85	20.05	4.32	30.33	6.84	36.8
PyramidKV2026.03		99.2	91	0	68.2	33.2	0	26.5	9.7	2.38	25.6	18.76	34.05
SnapKV2026.03		98.8	89	0	60.2	35.4	0	17.25	7.45	5.18	28.53	13.24	32.28
H2O2026.03		98.2	75.8	8	62	38.8	4.8	49.05	10.6	63.4	75.07	92.2	52.54
H2O2026.03		88.4	63.8	2.4	45	25.4	1.4	24.4	2.85	46.56	64.6	69.64	39.5
DapQ2026.03		85.2	87.4	0	26.2	19.6	0	3.65	1.35	0.78	0.33	3.6	20.74
H2O2026.03		67.2	56.8	2.4	25.4	15.6	0	9.05	1.25	31.1	48.2	20.64	25.24
SnapKV2026.03		58.8	65.4	0	20.4	14	0	0.75	0.4	0.14	0.07	2.28	14.75
PyramidKV2026.03		48.8	47.2	0	13.4	7.2	0	0.4	0.25	0.08	0	0.6	10.72
H2O2026.03		41.4	38.8	2.4	14.8	2.8	0	2.2	0.3	18.06	13	7.88	12.88
SLM2026.03		26.6	25.4	25.4	22.6	24.2	17.8	24.1	24.3	4.9	78.6	21.16	26.82
H2O2026.03		22	21.2	0	3.8	0.2	0	0.4	0.25	5.7	0.07	3.52	5.19
LaCache2026.03		21	27.4	1.8	29.6	29	3.2	12.2	6.45	86.08	86.07	11.04	28.53
SLM2026.03		13.4	13.8	11.4	11.4	13.2	10.8	10.95	11.3	0.38	75.07	8.16	16.35
SLM2026.03		3.6	6.2	5.6	6.6	6.2	5.4	6.25	6.75	0.18	75.33	1.08	11.2
SLM2026.03		1.2	1.2	1.2	2	3.4	2.4	2.3	2.45	0.14	78.6	0	8.63
SLM2026.03		0.6	1.2	1.2	2	2	0	2.25	2.45	0.2	44.93	0	5.17
LaCache2026.03		0.2	4.8	2.4	4.2	4	0	2.3	2.45	55.02	86.87	1.56	14.89
LaCache2026.03		0	0.2	0	2.6	0.4	0	0.05	1.4	16.52	78.8	0.08	9.1
LaCache2026.03		0	0	0	0	0	0	0	0	3.64	58.2	0	5.62
LaCache2026.03		0	0	0	0	0	0	0	0	0.58	8.4	0	0.82
LaCache2026.03		0	0	0	0	0	0	0	0	0.18	0.67	0	0.08
SLM2026.03		0	0	0	0	0	0	0	0	0.06	27.53	0	2.51