Synthetic Tasks on LongBench
14.67PCountSnapKV
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SnapKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 14.67 | 0.8667 | |
| DapQBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 14.17 | 0.8667 | |
| FullKVBackbone=Llama3-8B-Instruct, KV Cache Size=Full2026.03 | 13.3 | 0.8367 | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 13 | 0.8633 | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 12.78 | 0.8367 | |
| DapQBackbone=Qwen3-8B, KV Cache Size=642026.03 | 12.67 | 0.83 | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 12.44 | 0.82 | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 12.33 | 0.8367 | |
| H2OBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 12.33 | 0.8667 | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 12.11 | 0.8367 | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 12.11 | 0.84 | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 12.11 | 0.8167 | |
| FullKVBackbone=Qwen3-8B, KV Cache Size=Full2026.03 | 11.98 | 0.8667 | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 11.89 | 0.7767 | |
| DapQBackbone=Llama3-8B-Instruct, KV Cache Size=2562026.03 | 11.78 | 0.8367 | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 11.78 | 0.78 | |
| H2OBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 11.72 | 0.8667 | |
| DapQBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 11.67 | 0.86 | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Size=1282026.03 | 11.44 | 0.8233 | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 11.33 | 0.8367 | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 11.33 | 0.8433 | |
| FullKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=Full2026.03 | 11 | 0.8633 | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 11 | 0.86 | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 11 | 0.85 | |
| DapQBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 11 | 0.8067 | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 11 | 0.8667 | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 10.67 | 0.86 | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 10.67 | 0.86 | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=2562026.03 | 10.67 | 0.86 | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 10.67 | 0.8567 | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 10.67 | 0.8467 | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=1282026.03 | 10.67 | 0.85 | |
| H2OBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 10.67 | 0.85 | |
| PyramidKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 10.67 | 0.8033 | |
| SnapKVBackbone=Qwen2.5-7B-Instruct, KV Cache Size=642026.03 | 10.67 | 0.82 | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 10.33 | 0.7967 | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 9.67 | 0.8667 | |
| H2OBackbone=Qwen3-8B, KV Cache Size=642026.03 | 9.67 | 0.8667 | |
| H2OBackbone=Llama3-8B-Instruct, KV Cache Size=642026.03 | 7.72 | 0.8233 | |
| CompilerKVModel=Qwen2-7B-Instruct, KV cache size=1282026.02 | 6.5 | 0.705 | |
| CompilerKVModel=LLaMA-3-8B-Instruct, KV cache size=1282026.02 | 5.5 | 0.6925 | |
| CompilerKVModel=Mistral-7B-Instruct-v0.2, KV cache size=1282026.02 | 4.25 | 0.703 | |
| CompilerKVModel=InternLM-2.5-Chat-1M, KV cache size=1282026.02 | 0.92 | 0.97 |