Long-context retrieval and reasoning on RULER 16K
85.2RULER ScoreFull Cache
Evaluation Results
| Method | Links | |
|---|---|---|
| Full CacheBackbone=LLaMA-3.1, Cache Budget (L)=1282026.06 | 85.2 | |
| Full CacheBackbone=Qwen2-7B, Cache Budget (L)=1282026.06 | 80.5 | |
| MOMENTKVBackbone=LLaMA-3.1, Cache Budget (L)=1282026.06 | 77.2 | |
| Ada-KVBackbone=LLaMA-3.1, Cache Budget (L)=1282026.06 | 73.8 | |
| MOMENTKVBackbone=Qwen2-7B, Cache Budget (L)=1282026.06 | 72 | |
| SnapKVBackbone=LLaMA-3.1, Cache Budget (L)=1282026.06 | 71.5 | |
| PyramidKVBackbone=LLaMA-3.1, Cache Budget (L)=1282026.06 | 70.2 | |
| Ada-KVBackbone=Qwen2-7B, Cache Budget (L)=1282026.06 | 68.5 | |
| SnapKVBackbone=Qwen2-7B, Cache Budget (L)=1282026.06 | 66.2 | |
| PyramidKVBackbone=Qwen2-7B, Cache Budget (L)=1282026.06 | 65.5 | |
| H2OBackbone=LLaMA-3.1, Cache Budget (L)=1282026.06 | 62.8 | |
| H2OBackbone=Qwen2-7B, Cache Budget (L)=1282026.06 | 58.5 |