Long-context Evaluation on LongBench
55.26Average ScoreIndexMem
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| IndexMemCompression ratio=50%2026.05 | 55.26 | — | — | — | — | — | — | — | |
| IndexMemCompression ratio=10%2026.05 | 53.48 | — | — | — | — | — | — | — | |
| IndexMemCompression ratio=25%2026.05 | 53.18 | — | — | — | — | — | — | — | |
| FullKVBackbone=Qwen3-8B, KV Cache Size=Full2026.03 | 52.6 | — | — | — | — | — | — | — | |
| FullKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=Full2026.03 | 50.67 | — | — | — | — | — | — | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 49.28 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 48.92 | — | — | — | — | — | — | — | |
| FullKVBackbone=Llama3-8B-Instruct, KV Cache Size=Full2026.03 | 48.39 | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 47.47 | — | — | — | — | — | — | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 46.95 | — | — | — | — | — | — | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 46.45 | — | — | — | — | — | — | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 46.4 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 46.19 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 46.17 | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 46.12 | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 45.7 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 45.61 | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 44.82 | — | — | — | — | — | — | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 44.4 | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 44.31 | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 44.22 | — | — | — | — | — | — | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 44.02 | — | — | — | — | — | — | — | |
| Expected AttentionCompression ratio=25%2026.05 | 43.88 | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 43.82 | — | — | — | — | — | — | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 43.82 | — | — | — | — | — | — | — | |
| TOVACompression ratio=10%2026.05 | 43.81 | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 43.78 | — | — | — | — | — | — | — | |
| Expected AttentionCompression ratio=10%2026.05 | 43.78 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 43.68 | — | — | — | — | — | — | — | |
| PyramidKVCompression ratio=10%2026.05 | 43.63 | — | — | — | — | — | — | — | |
| FP16BIT=162026.07 | 43.59 | 22.9 | 47.39 | 54.22 | 22.6 | 30.31 | 88.04 | 39.67 | |
| SnapKVCompression ratio=10%2026.05 | 43.55 | — | — | — | — | — | — | — | |
| HeadKV w/ CriticalKVBackbone=Mistral-7B, Cache Budget=20%2025.02 | 43.46 | — | — | — | — | — | — | — | |
| SnapKVCompression ratio=25%2026.05 | 43.37 | — | — | — | — | — | — | — | |
| DapQBackbone=Qwen3-8B, KV Cache Size=642026.03 | 43.23 | — | — | — | — | — | — | — | |
| TOVACompression ratio=25%2026.05 | 43.08 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 43.02 | — | — | — | — | — | — | — | |
| AdaKV w/ CriticalKVBackbone=Mistral-7B, Cache Budget=20%2025.02 | 42.85 | — | — | — | — | — | — | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 42.79 | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen3-8B, KV Cache Size=642026.03 | 42.47 | — | — | — | — | — | — | — | |
| PyramidKVCompression ratio=50%2026.05 | 42.35 | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 42.27 | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 42.22 | — | — | — | — | — | — | — | |
| IndexMemCompression ratio=75%2026.05 | 42.18 | — | — | — | — | — | — | — | |
| TOVACompression ratio=50%2026.05 | 42.1 | — | — | — | — | — | — | — | |
| Expected AttentionCompression ratio=50%2026.05 | 42.09 | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 42.03 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 41.88 | — | — | — | — | — | — | — | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 41.81 | — | — | — | — | — | — | — | |
| SnapKV w/ CriticalKVBackbone=Mistral-7B, Cache Budget=20%2025.02 | 41.77 | — | — | — | — | — | — | — | |
| PyramidKVCompression ratio=25%2026.05 | 41.57 | — | — | — | — | — | — | — | |
| SnapKVCompression ratio=50%2026.05 | 41.44 | — | — | — | — | — | — | — | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 41.33 | — | — | — | — | — | — | — | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 40.96 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 40.78 | — | — | — | — | — | — | — | |
| SnapKV w/ OBCache (Joint)Backbone=Mistral-7B, Cache Budget=20%2025.02 | 40.49 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Mistral-7B, Cache Budget=20%2025.02 | 40.11 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 40.09 | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 39.98 | — | — | — | — | — | — | — | |
| GSRQBIT=2.002026.07 | 39.39 | 21.96 | 44.88 | 48.24 | 22.67 | 24.42 | 75.3 | 38.27 | |
| SnapKVCompression ratio=75%2026.05 | 39.22 | — | — | — | — | — | — | — | |
| PyramidKVCompression ratio=75%2026.05 | 39.17 | — | — | — | — | — | — | — | |
| TOVACompression ratio=75%2026.05 | 39.05 | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 38.9 | — | — | — | — | — | — | — | |
| VQLLMBIT=2.002026.07 | 38.47 | 23.38 | 42.2 | 46.46 | 22.14 | 21.14 | 77.12 | 36.85 | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 38.45 | — | — | — | — | — | — | — | |
| CompilerKVModel=LLaMA-3-8B-Instruct, KV cache size=1282026.02 | 37.97 | — | — | — | — | — | — | — | |
| CompilerKVModel=Mistral-7B-Instruct-v0.2, KV cache size=1282026.02 | 37.63 | — | — | — | — | — | — | — | |
| IndexMemCompression ratio=90%2026.05 | 36.51 | — | — | — | — | — | — | — | |
| CompilerKVModel=InternLM-2.5-Chat-1M, KV cache size=1282026.02 | 36.5 | — | — | — | — | — | — | — | |
| Expected AttentionCompression ratio=75%2026.05 | 36.33 | — | — | — | — | — | — | — | |
| CompilerKVModel=Qwen2-7B-Instruct, KV cache size=1282026.02 | 36.13 | — | — | — | — | — | — | — | |
| TOVACompression ratio=90%2026.05 | 32.56 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Llama3.1-8B, KV Cache Size=2562026.03 | 31.96 | — | — | — | — | — | — | — | |
| FullKVBackbone=Llama3.1-8B2026.03 | 31.63 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Llama3.1-8B, KV Cache Size=5122026.03 | 31.39 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Llama3.1-8B, KV Cache Size=1282026.03 | 31.32 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Llama3.1-8B, KV Cache Size=10242026.03 | 31.14 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Llama3.1-8B, KV Cache Size=20482026.03 | 31.01 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Llama3.1-8B, KV Cache Size=642026.03 | 30.62 | — | — | — | — | — | — | — | |
| PyramidKVCompression ratio=90%2026.05 | 29.68 | — | — | — | — | — | — | — | |
| SnapKVCompression ratio=90%2026.05 | 29.4 | — | — | — | — | — | — | — | |
| Expected AttentionCompression ratio=90%2026.05 | 28.6 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=20482026.03 | 26.76 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 26.06 | — | — | — | — | — | — | — | |
| FullKVBackbone=Qwen3-8B2026.03 | 26.04 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 25.88 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=10242026.03 | 25.36 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=5122026.03 | 25.33 | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 22.11 | — | — | — | — | — | — | — | |
| GSRQBIT=1.002026.07 | 20.21 | 18.56 | 30.28 | 40.77 | 21.48 | 11.88 | 8.17 | 10.35 | |
| GSRQBIT=0.752026.07 | 14.96 | 12.39 | 17.42 | 29.7 | 21.11 | 5.46 | 7.91 | 8.83 | |
| VQLLMBIT=1.002026.07 | 12.23 | 6.99 | 15.73 | 24.41 | 18.35 | 3.73 | 11.43 | 4.96 | |
| MambaTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 4.96 | — | — | — | — | — | — | — | |
| TransformerTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 4.44 | — | — | — | — | — | — | — | |
| SRMTraining=compute-matched, Training dataset=FineWeb-edu2026.05 | 4.09 | — | — | — | — | — | — | — |