Long-context language understanding on LongBench (Aggregate Score Only)
49.8Aggregate ScoreFullKV
Evaluation Results
| Method | Links | |
|---|---|---|
| FullKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=Full2026.06 | 49.8 | |
| FullKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=Full2026.06 | 49.08 | |
| CompressKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=10242026.06 | 48.69 | |
| FullKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=Full2026.06 | 48.57 | |
| CAKELLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=10242026.06 | 48.52 | |
| HeadKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=10242026.06 | 48.42 | |
| CompressKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=10242026.06 | 48.24 | |
| AdaKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=10242026.06 | 48.19 | |
| SnapKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=10242026.06 | 48.18 | |
| CAKELLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=10242026.06 | 47.97 | |
| AdaKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=10242026.06 | 47.94 | |
| FullKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=Full2026.06 | 47.82 | |
| SnapKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=10242026.06 | 47.82 | |
| CompressKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=10242026.06 | 47.78 | |
| PyramidKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=10242026.06 | 47.7 | |
| PyramidKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=10242026.06 | 47.65 | |
| CAKELLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=10242026.06 | 47.51 | |
| HeadKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=10242026.06 | 47.5 | |
| SnapKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=10242026.06 | 47.34 | |
| AdaKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=10242026.06 | 47.33 | |
| HeadKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=10242026.06 | 47.05 | |
| PyramidKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=10242026.06 | 46.98 | |
| CompressKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=10242026.06 | 46.96 | |
| CompressKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=2562026.06 | 46.71 | |
| CAKELLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=10242026.06 | 46.66 | |
| SnapKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=10242026.06 | 46.48 | |
| HeadKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=10242026.06 | 46.41 | |
| AdaKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=10242026.06 | 46.38 | |
| CAKELLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=2562026.06 | 46.3 | |
| PyramidKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=10242026.06 | 45.96 | |
| CompressKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=2562026.06 | 45.43 | |
| CompressKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=2562026.06 | 45.37 | |
| SnapKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=2562026.06 | 45.21 | |
| CAKELLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=2562026.06 | 44.73 | |
| CompressKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=2562026.06 | 44.73 | |
| CAKELLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=2562026.06 | 44.7 | |
| CAKELLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=2562026.06 | 44.49 | |
| AdaKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=2562026.06 | 44.45 | |
| PyramidKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=2562026.06 | 44.36 | |
| HeadKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=2562026.06 | 44.21 | |
| HeadKVLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=2562026.06 | 44.11 | |
| HeadKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=2562026.06 | 44.1 | |
| HeadKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=2562026.06 | 44.02 | |
| SnapKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=2562026.06 | 43.77 | |
| SnapKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=2562026.06 | 43.76 | |
| AdaKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=2562026.06 | 43.75 | |
| AdaKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=2562026.06 | 43.68 | |
| SnapKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=2562026.06 | 43.36 | |
| AdaKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=2562026.06 | 43.3 | |
| PyramidKVLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=2562026.06 | 43.06 | |
| PyramidKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=2562026.06 | 42.71 | |
| PyramidKVLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=2562026.06 | 42.11 | |
| StreamingLLMLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=10242026.06 | 36.95 | |
| StreamingLLMLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=10242026.06 | 34.73 | |
| StreamingLLMLLM Scale=Small-scale LLMs, Backbone=Llama-3.1-8B, KV-cache budget=2562026.06 | 33.92 | |
| StreamingLLMLLM Scale=Small-scale LLMs, Backbone=Mistral-7B, KV-cache budget=2562026.06 | 31.22 | |
| StreamingLLMLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=10242026.06 | 29.84 | |
| StreamingLLMLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=10242026.06 | 29.62 | |
| StreamingLLMLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-14B, KV-cache budget=2562026.06 | 25.86 | |
| StreamingLLMLLM Scale=Large-scale LLMs, Backbone=Qwen2.5-32B, KV-cache budget=2562026.06 | 25.28 |