Long-context language understanding on Ruler (Avg.)
70.12Ruler Avg ScoreHeadKV w/ CriticalKV
Evaluation Results
| Method | Links | |
|---|---|---|
| HeadKV w/ CriticalKVBackbone=Llama-3.1-8B, Cache budget=20%2025.02 | 70.12 | |
| AdaKV w/ CriticalKVBackbone=Llama-3.1-8B, Cache budget=20%2025.02 | 68.94 | |
| SnapKV w/ CriticalKVBackbone=Llama-3.1-8B, Cache budget=20%2025.02 | 58.7 | |
| SnapKV w/ OBCache (Joint)Backbone=Llama-3.1-8B, Cache budget=20%2025.02 | 58.47 | |
| SnapKVBackbone=Llama-3.1-8B, Cache budget=20%2025.02 | 54.9 |