Long-context understanding on LongBench-E 1.0 (test)

44.56Qasper ScoreExact (Baseline)

Evaluation results

Method
Exact (Baseline)2025.02	44.56	50.65	69.14	60.39	21.3	19.52	75.33	81.14	43.17	22	99.67	50.9	35.22	51.77
Exact (Baseline)2025.02	43.39	52.63	64.06	53.71	28.07	22.4	74.67	88.75	44.81	22.33	99	63.69	46.3	54.14
Exact (Baseline)2025.02	42.87	48.54	52.05	38.6	31.31	22.07	71.67	91.85	42.36	20.37	98.13	49.62	42.73	50.17
BalanceKV2025.02	40.14	43.17	64.46	58.06	22.26	20.32	73	80.68	41.07	22.33	92	44.95	32.43	48.84
Uniform2025.02	39.28	43.82	64.82	57.84	23.1	19.5	73	81.63	39.7	22	92	44.97	32.19	48.76
Uniform2025.02	37.18	37.15	47.49	37.82	27.56	21.06	68.67	90.48	36.13	20.33	96.26	45.72	37.09	46.38
Uniform2025.02	37.07	41.69	61.11	49.96	29.18	22.67	71.67	87.89	40.34	22	84.33	58	42.57	49.88
BalanceKV2025.02	37.02	41.96	61.74	50.9	29.26	22.64	71.67	88.1	41.14	23.67	87.67	58.64	43.63	50.62
SnapKV2025.02	36.21	46.78	66.64	57.02	16.35	16.07	70.33	77.53	41.08	22	99.33	49.04	35.62	48.77
BalanceKV2025.02	35.75	37.04	46.37	36.24	27.09	20.84	69	90.88	37.88	20.39	96.65	48.45	41.4	46.77
PyramidKV2025.02	34.47	46.33	67.78	55.92	15.16	15.39	69.33	63.24	40.36	22.67	99.33	48.32	34.35	47.13
SnapKV2025.02	33.91	42.55	49.09	36.13	20.48	17.67	62	91.7	40.23	20.33	98.86	46.7	39.86	46.12
PyramidKV2025.02	33.86	39.75	47.12	35.96	20.03	17.78	63.67	90.76	40.21	20.4	98.97	45.25	39.51	45.64
SnapKV2025.02	32.95	47.53	61.96	50.34	20.29	18.28	60.67	88.75	42.97	22.67	99.33	61.32	45.84	50.22
PyramidKV2025.02	31.76	46.6	62.83	50	19.08	17.68	65	85.52	42.61	22	99.33	60.62	44.4	49.8
StreamingLLM2025.02	20.73	32.62	49.93	42.39	21.63	18.69	59.67	74.96	29.57	11.67	63	46.16	32.25	38.71
StreamingLLM2025.02	20.65	30.71	39.14	32.43	23.1	18.7	58	83.87	28.85	20.36	97.26	33.69	30.46	39.79
StreamingLLM2025.02	20.12	34.35	51.84	48.23	19.09	17.1	61	51.14	28.52	23.33	41.33	39.19	26.54	35.52