Long-context language understanding on LongBench 1.0 (test)

61.5MultiNewsOriginal

Evaluation Results

Method
Original2025.02	61.5	—	77.8	40.7	9.8	—	—	—	—	—	31.52	18.7	19.2	36.8	25.4	32.8	27.3	20.8	25.8	52.4	43.8
LongLora-7B-16k2024.01	27.7	0.5	63.5	85.7	41.9	26	57.6	54.5	39.79	14.55	34.18	—	—	—	—	—	—	—	—	—	—
EntropyInfer2026.06	27.45	—	73	—	43.94	—	63.23	56.47	—	—	48.91	30.2	45.38	55.2	55.37	44.25	31.68	35.18	25.27	5.97	98.5
Base2026.06	27.3	—	72.5	—	43.66	—	63.04	56.67	—	—	48.94	29.71	45.3	55.13	55.5	44.3	31.32	35.16	25.31	6.96	99.5
CritiPrefill2026.06	27.26	—	72	—	44.18	—	63.24	55.55	—	—	48.32	27.29	45.43	55.46	56.54	43.62	29.38	35.04	24.96	4.64	97.5
Vicuna-v1.5-7B-16k2024.01	27.2	15.1	74	86.2	40.8	28.8	51	43.5	36.49	26.55	34.28	—	—	—	—	—	—	—	—	—	—
Vicuna-v1.5-13B-16k2024.01	27.1	16.4	74	84.9	27.8	29.8	44.1	45.6	38.08	23.86	34.92	—	—	—	—	—	—	—	—	—	—
E2-LLM-Llama2-13B-16k2024.01	27	9.8	73.5	87.9	40.6	36	65.4	59.1	44.73	28.56	41.13	—	—	—	—	—	—	—	—	—	—
H2O2026.04	26.81	—	69	91.15	43.97	—	59.27	53.48	—	—	36.49	22.94	12.61	27.34	16.63	15.81	10.14	33.51	23.47	6.66	71
E2-LLM-Llama2-13B-32k2024.01	26.8	10.2	75	87.8	40.9	44.5	63.8	57.5	44.55	31.93	41.74	—	—	—	—	—	—	—	—	—	—
GPT-3.5-Turbo-16k2024.01	26.7	16	68	91.4	41.7	29.2	54.7	53.6	44.6	33.78	42.19	—	—	—	—	—	—	—	—	—	—
LongChat-v1.5-7B-32k2024.01	26.4	9.9	63.5	82.3	34.2	23.2	53	55.3	36.86	20.43	33.21	—	—	—	—	—	—	—	—	—	—
EntropyInfer2026.06	26.21	—	73.5	—	44.23	—	62.77	56.2	—	—	48.55	29.86	45.18	54.74	55.28	45.88	31.74	30.02	24.97	6.49	98.5
Llama2-13B-chat-4k2024.01	26.1	17.2	66	85.2	36.5	20.3	51.9	52.8	37.87	24.38	34.87	—	—	—	—	—	—	—	—	—	—
SnapKV2026.06	26.05	—	67.5	—	42.67	—	62.13	56.24	—	—	47.66	29.64	43.89	54.12	54.72	44.81	29.99	28.62	24.2	6.82	99.5
AdaKV2026.06	26	—	68.5	—	42.4	—	62.33	56.35	—	—	47.68	29.53	42.91	53.7	55.23	44.84	30.62	28.36	24.11	7.01	99.5
Full-context2026.04	25.91	—	73.5	88.93	47.92	—	62.03	66.83	—	—	49.43	28.79	44.79	51.77	58.38	46.11	29.23	33.46	24.32	8.83	100
SnapKV2026.04	25.91	—	72	88.56	47.68	—	61.45	56.84	—	—	48.25	28.13	44.79	51.21	55.83	45.82	28.24	33.46	23.85	8.76	99.5
E2-LLM-Llama2-7B-16k2024.01	25.9	9.6	68.5	89.2	38.2	35	65.8	58.1	41.26	26.7	38.03	—	—	—	—	—	—	—	—	—	—
PI-Llama2-13B-16k2024.01	25.9	9.1	72.5	86.5	27.9	31	62.5	51.1	40.88	26.35	37.65	—	—	—	—	—	—	—	—	—	—
SnapKV2026.04	25.83	—	70.5	87.65	47.53	—	61.45	56.42	—	—	47.75	27.68	44.79	50.12	55.72	45.15	27.13	33.25	23.2	8.15	99.5
Llama2-7B-chat-4k2024.01	25.8	0.2	61.5	77.8	40.7	19.8	52.4	43.8	35.17	15.45	20.79	—	—	—	—	—	—	—	—	—	—
StreamingLLM2026.04	25.73	—	72	89.41	47.73	—	61.06	56.75	—	—	46.39	25.85	44.79	49.1	50.73	44.92	24.83	33.2	23.2	7.91	85
StreamingLLM2026.04	25.73	—	72.5	89.96	47.73	—	62.07	57.85	—	—	46.92	27.99	44.79	49.23	51.64	45.12	25.94	33.25	23.45	8.47	85
StructKV2026.04	25.67	—	73	88.67	47.82	—	61.78	61.87	—	—	48.89	28.45	44.79	51.25	57.87	45.78	28.78	33.34	24.32	8.83	100
FullKV2026.03	25.64	—	70	89.85	40.55	—	58.96	52.71	—	—	48.39	—	37.72	40.64	50.17	34.88	—	31.03	—	13.3	83.67
H2O2026.04	25.63	—	69	91.05	44.41	—	62.17	54.16	—	—	37.88	23.48	15.78	29.43	19.21	20.47	15.13	32.47	21.58	7.12	75
StructKV2026.04	25.53	—	72	88.12	47.26	—	61.67	60.12	—	—	48.39	28.34	44.79	50.67	56.76	45.61	28.21	32.12	24.32	8.71	100
E2-LLM-Llama2-7B-32k2024.01	25.4	11.7	70.5	88.4	32.5	40	64.5	60.9	41.74	30.23	39.18	—	—	—	—	—	—	—	—	—	—
Original2025.02	24.78	—	67.5	74.24	40.28	—	56.39	50.36	—	—	27.22	8.48	13.97	20.4	13.62	16.77	5.46	25.17	12.47	2.3	3.25
ChunkLlama-16k2025.02	24.78	—	67.5	74.24	40.28	—	56.39	50.36	—	—	27.22	8.48	13.97	20.4	13.62	16.77	5.46	25.17	12.47	2.3	3.25
FastKV2026.04	24.67	—	72	87.54	45.87	—	61.17	58.87	—	—	47.55	28.24	44.79	50.36	51.34	44.23	26.87	33.32	24.32	7.65	99.5
FastKV2026.04	24.18	—	71	86.21	45.62	—	60.22	57.84	—	—	46.99	28.12	44.79	50.12	50.22	43.13	26.25	33.12	24.28	7.22	99.5
GemFilter2026.04	24.16	—	71	88.56	44.16	—	60.12	46.78	—	—	45.02	24.13	44.21	45.45	49.56	44.57	27.12	32.19	23.11	6.22	89
GemFilter2026.04	24.13	—	70.5	88.43	43.58	—	59.65	44.58	—	—	44.22	24.01	43.68	44.27	48.92	43.88	25.23	31.49	22.3	5.88	87
Base2026.06	23.96	—	72	—	45.64	—	60.07	67.26	—	—	48.82	28.73	44.21	53.05	57.98	46.33	30.03	31.86	23.23	8	100
EntropyInfer2026.06	23.82	—	71	—	46.23	—	60.4	65.53	—	—	47.98	26.81	44.14	52.13	57.3	47.16	30.92	31.48	23.6	6.5	91.5
CritiPrefill2026.06	23.76	—	70	—	46.76	—	60.27	65.63	—	—	47.34	25.44	44.2	51.19	54.65	45.92	27.81	31.45	23.19	7	91
FullKV2026.03	23.69	—	71.67	91.79	42.07	—	70.64	59.26	—	—	52.6	—	36.87	53.67	57.67	44.73	—	33.39	—	11.98	86.67
EntropyInfer2026.06	23.38	—	71	—	46.22	—	60.3	65.26	—	—	47.68	26.81	43.98	51.73	57.2	47.09	30.85	28.74	23.18	6.5	91.5
GALI-16k2025.02	23.37	—	62	72.8	25.12	—	58.71	48.51	—	—	25.8	6.29	16.73	22.26	5.88	26.8	12.82	23.58	15.96	1.83	2.83
SnapKV2026.06	22.75	—	67.5	—	45.03	—	59.75	65.19	—	—	47.43	28.57	42.61	51.34	57.16	44.54	29.54	27.05	23.11	8	100
AdaKV2026.06	22.39	—	68.5	—	44.91	—	59.55	65.75	—	—	47.51	28.43	42.2	51.86	56.95	44.54	29.77	26.6	23.11	8	100
DapQ2026.03	21.89	—	60.67	90.48	38.34	—	62.78	55.64	—	—	46.4	—	32.55	38.18	50.67	34.35	—	22.25	—	11.78	83.67
PyramidKV2026.03	21.84	—	60	89.26	37.07	—	61.34	52.51	—	—	45.7	—	30.88	38.11	50.2	33.88	—	22.54	—	12.78	83.67
SnapKV2026.03	21.53	—	57	89.65	36.97	—	61.78	54.92	—	—	45.61	—	30.84	38.39	49.75	33.8	—	22.18	—	12.11	84
H2O2026.03	21.44	—	45.67	89.49	38.28	—	61.49	53.36	—	—	44.02	—	28.11	36.63	48.62	31.5	—	21.87	—	12.11	83.67
SelfExtend-16k2025.02	21.35	—	25.78	69.5	81.99	—	60.6	54.33	—	—	34.62	16.77	21.69	25.02	35.21	34.34	30.24	14.13	27.32	5.66	5.83
DapQ2026.03	20.63	—	50	90.06	36.87	—	61.81	53.92	—	—	44.4	—	28.76	37.24	50.04	—	—	20.47	—	12.11	81.67
PyramidKV2026.03	20.55	—	49	87.68	36.73	—	60.36	52.03	—	—	43.82	—	28.8	38.29	49.52	31.6	—	20.67	—	12.44	82
H2O2026.03	20.3	—	40	87.29	36.25	—	59.81	53.14	—	—	42.79	—	25.95	36.25	48.65	31.9	—	20.79	—	12.33	83.67
SnapKV2026.03	20.08	—	47.67	87.82	35.63	—	61.49	52.4	—	—	43.68	—	29.52	37.8	49.36	32.4	—	19.87	—	11.44	82.33
H2O2026.03	19.14	—	37.33	86.27	35.18	—	59.2	51.1	—	—	40.96	—	24.02	30.83	48.27	31.7	—	19.37	—	7.72	82.33
DapQ2026.03	19.01	—	62.67	91.15	39.61	—	67.2	53.83	—	—	49.28	—	32.14	50.78	54.79	44.47	—	24.16	—	14.17	86.67
SnapKV2026.03	18.99	—	63	91.07	38.57	—	67.99	53.77	—	—	48.92	—	32.4	49.29	54.38	41.4	—	23.79	—	14.67	86.67
PyramidInfer2026.04	18.94	—	67	87.12	45.62	—	60.24	57.82	—	—	41.95	25.61	36.76	40.12	41.12	41.45	20.12	26.34	20.78	4.12	78
DapQ2026.03	18.7	—	38.67	87.38	35.3	—	60.19	49.9	—	—	41.81	—	25.99	37.36	49.11	32.88	—	18.46	—	11.89	77.67
H2O2026.03	18.26	—	46.33	90.04	38.98	—	66.11	53.93	—	—	46.12	—	27.8	44.92	51.37	40.5	—	22.38	—	12.33	86.67
PyramidKV2026.03	17.41	—	61.67	90.96	37.65	—	67.01	50.11	—	—	47.47	—	30.41	47.81	51	41.37	—	22.36	—	13	86.33
H2O2026.03	17.27	—	41.33	90.21	38.16	—	65.22	52.77	—	—	44.22	—	26.6	41.37	48.1	39.85	—	20.83	—	11.72	86.67
DapQ2026.03	17.27	—	54	90.45	38.5	—	66.29	52.03	—	—	46.95	—	29.1	47.15	53.84	43	—	21.11	—	11.67	86
SnapKV2026.03	17.09	—	40.67	86.02	33.99	—	57.95	50.91	—	—	40.78	—	25.06	32.92	47.16	31.71	—	16.85	—	11.78	78
SnapKV2026.03	16.64	—	51.33	91.07	37.37	—	65.36	51.65	—	—	46.17	—	29.41	46.43	51.2	41.66	—	20.37	—	11	86.67
PyramidKV2026.03	16.34	—	39	76.8	32.31	—	55.19	47.9	—	—	38.9	—	22.04	31.8	47.01	31.54	—	15.7	—	10.33	79.67
H2O2026.03	15.23	—	39	88.13	35.98	—	59.48	48.95	—	—	42.47	—	25.55	38.94	46.66	39.27	—	18.55	—	9.67	86.67
PyramidKV2026.03	15.1	—	48.33	89.31	36.92	—	60.82	48.78	—	—	43.78	—	26.22	40.48	48.41	39.46	—	18.99	—	9.67	86.67
DapQ2026.03	13.95	—	43	88.97	36	—	60.31	46.9	—	—	43.23	—	25.78	43.17	49.84	41.28	—	17.16	—	12.67	83
SelfExtend-16k2025.02	13.65	—	6.31	67.5	69.88	—	58.21	51.65	—	—	26.97	6.89	12.67	25.95	9.08	11.25	7.91	16.39	22.79	2.18	3.21
PyramidKV2026.03	12.93	—	44.67	88.27	34.63	—	59.73	46.48	—	—	42.27	—	25.32	40.44	46.61	39.2	—	16.25	—	11.33	83.67
SnapKV2026.03	12.38	—	42.67	87.93	35.12	—	57.96	46.49	—	—	41.88	—	25.09	39.89	46.58	39.38	—	15.28	—	11.33	84.33
Dyn-NTK-16k2025.02	10.9	—	25.92	64.5	40.6	—	57.13	42.16	—	—	23.57	3.79	10.37	22.38	10.26	26.13	13.36	1.32	24.78	2.29	4.33
YaRN-16k2025.02	6.31	—	63.5	45.35	32.36	—	51.74	43.91	—	—	22.22	3.22	10.86	22.14	3.81	29.52	20.13	22.84	13.65	2.2	2.15
NTK-16k2025.02	0.51	—	50.7	27.91	17.08	—	56.39	50.36	—	—	17.08	0.73	10.33	19.44	0.42	19.47	6.26	23.17	0.52	2.3	3.25