Long-context language understanding on SCBench
79KV RetrievalLlama-3.1-8B
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Llama-3.1-8BBackbone=Llama-3.1-8B, KR=100, CR=1002026.02 | 79 | 21.7 | 56.8 | 44.1 | 50.4 | — | — | — | — | — | — | — | |
| OmniKVBackbone=Llama-3.1-8B, KR=100, CR=302026.02 | 72.2 | 21.3 | 57 | 46.3 | 49.2 | — | — | — | — | — | — | — | |
| Qwen2.5-7BBackbone=Qwen2.5-7B, KR=100, CR=1002026.02 | 70.4 | 22.9 | 60.7 | 57 | 52.8 | — | — | — | — | — | — | — | |
| OmniKVBackbone=Qwen2.5-7B, KR=48, CR=302026.02 | 69.2 | 22.7 | 61.2 | 56.7 | 52.4 | — | — | — | — | — | — | — | |
| Full KVBackbone=Qwen3, KV budget=Full2025.12 | 66 | 28.78 | — | — | — | 20.52 | 95.57 | 32.6 | 53.86 | 36.48 | 38.33 | 49.6 | |
| DeltaKV†Backbone=Qwen2.5-7B, KR=48, CR=302026.02 | 62.4 | 23.4 | 61.3 | 58.2 | 51.3 | — | — | — | — | — | — | — | |
| DeltaKV†Backbone=Llama-3.1-8B, KR=45, CR=302026.02 | 60.4 | 19.5 | 52.4 | 53 | 46.3 | — | — | — | — | — | — | — | |
| DeltaKV + 4-bitBackbone=Llama-3.1-8B, KR=29, CR=30, quantization=4-bit2026.02 | 60.4 | 20.7 | 52.2 | 53.7 | 46.8 | — | — | — | — | — | — | — | |
| DeltaKVBackbone=Qwen2.5-7B, KR=48, CR=302026.02 | 59.4 | 24 | 60.6 | 58.5 | 50.6 | — | — | — | — | — | — | — | |
| DeltaKVBackbone=Llama-3.1-8B, KR=45, CR=302026.02 | 58 | 20.5 | 50 | 51.5 | 45 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B, KR=30, CR=302026.02 | 6.2 | 21.3 | 60.6 | 56.3 | 36.1 | — | — | — | — | — | — | — | |
| StreamingLLMBackbone=Qwen3, KV budget=40962025.12 | 2.2 | 6.84 | — | — | — | 5.68 | 100 | 13.2 | 2.96 | 29.21 | 28.25 | 0 | |
| SnapKVBackbone=Llama-3.1-8B, KR=30, CR=302026.02 | 0.4 | 20.3 | 55.7 | 43.7 | 30 | — | — | — | — | — | — | — | |
| H2OBackbone=Qwen3, KV budget=40962025.12 | 0 | 3.7 | — | — | — | 4.8 | 100 | 8 | 0.46 | 8.97 | 6.51 | 0.27 | |
| SnapKVBackbone=Qwen3, KV budget=40962025.12 | 0 | 6.29 | — | — | — | 10.04 | 100 | 18.6 | 0.23 | 27.9 | 29.28 | 0.31 | |
| TRIM-KVBackbone=Qwen3, KV budget=40962025.12 | 0 | 19.09 | — | — | — | 13.1 | 100 | 13.8 | 4.32 | 33.66 | 34.06 | 43.11 |