Summarization on LongBench
33.39GovRep ScoreFullKV
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FullKVBackbone=Qwen3-8B, KV Cache Size=Full2026.03 | 33.39 | — | 23.69 | — | |
| FullKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=Full2026.03 | 31.64 | — | 22.84 | — | |
| LingoEDUBackbone=GPT-4.1, Approach=EDU-based context compression2025.12 | 31.56 | 23.3 | 23.5 | 14.62 | |
| FullKVBackbone=Llama3-8B-Instruct, KV Cache Size=Full2026.03 | 31.03 | — | 25.64 | — | |
| Self-SumBackbone=GPT-4.1, Approach=LLM-generated abstractive summary2025.12 | 30.98 | 22.53 | 22.06 | 13.71 | |
| StandardBackbone=GPT-4.1, Approach=Full context2025.12 | 29.97 | 22.84 | 20.85 | 12.5 | |
| GlyphApproach=Implicit compression2025.12 | 25.53 | 19.78 | — | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 24.16 | — | 19.01 | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 23.79 | — | 18.99 | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 22.54 | — | 21.84 | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 22.38 | — | 18.26 | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 22.36 | — | 17.41 | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 22.35 | — | 18.4 | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 22.25 | — | 21.89 | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 22.25 | — | 18.41 | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 22.18 | — | 21.53 | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 22.13 | — | 18.31 | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 21.87 | — | 21.44 | — | |
| CompilerKVModel=Qwen2-7B-Instruct, KV cache size=1282026.02 | 21.62 | 22.47 | 19.74 | — | |
| CompilerKVModel=Mistral-7B-Instruct-v0.2, KV cache size=1282026.02 | 21.55 | 23.21 | 22.35 | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 21.11 | — | 17.27 | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 20.83 | — | 17.27 | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 20.79 | — | 20.3 | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 20.67 | — | 20.55 | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 20.47 | — | 20.63 | — | |
| CompilerKVModel=LLaMA-3-8B-Instruct, KV cache size=1282026.02 | 20.43 | 22.16 | 21.35 | — | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 20.42 | — | 16.8 | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 20.37 | — | 16.64 | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 20.13 | — | 16.64 | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 19.89 | — | 16.68 | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 19.87 | — | 20.08 | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 19.42 | — | 16.19 | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 19.37 | — | 19.14 | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 18.99 | — | 15.1 | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=642026.03 | 18.55 | — | 15.23 | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 18.46 | — | 18.7 | — | |
| C3Approach=Latent-based compression2025.12 | 18.2 | 7.35 | 18.03 | 0.39 | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 18.01 | — | 15.15 | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 17.96 | — | 15.05 | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=642026.03 | 17.16 | — | 13.95 | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 16.9 | — | 14.05 | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 16.85 | — | 17.09 | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 16.25 | — | 12.93 | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 15.7 | — | 16.34 | — | |
| CompilerKVModel=InternLM-2.5-Chat-1M, KV cache size=1282026.02 | 15.5 | 22.62 | 18.46 | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 15.42 | — | 12.03 | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 15.28 | — | 12.38 | — | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 14.04 | — | 11.1 | — | |
| Self-SumBackbone=Gemini-2.5-Pro, Approach=LLM-generated abstractive summary2025.12 | 4.34 | 16.53 | 4.44 | 6.17 | |
| LingoEDUBackbone=Gemini-2.5-Pro, Approach=EDU-based context compression2025.12 | 4.25 | 16.17 | 4.85 | 6.36 | |
| StandardBackbone=Gemini-2.5-Pro, Approach=Full context2025.12 | 4.1 | 15.8 | 4.05 | 5.8 |