Long-context evaluation on RULER 128k Zero-shot transfer
31.3Accuracy (RULER 128k Zero-shot)Still
Evaluation results
| Method | Links | |
|---|---|---|
| StillK (Cache budget)=1024, Number of seeds=32026.06 | 31.3 | |
| StillK (Cache budget)=2048, Number of seeds=32026.06 | 30.4 | |
| StillK (Cache budget)=512, Number of seeds=32026.06 | 25.8 | |
| StillK (Cache budget)=328, Number of seeds=32026.06 | 23.4 | |
| KV-DistillK (Cache budget)=2048, Number of seeds=32026.06 | 22 | |
| KV-DistillK (Cache budget)=164, Number of seeds=32026.06 | 20.7 | |
| StillK (Cache budget)=164, Number of seeds=32026.06 | 20.5 | |
| KV-DistillK (Cache budget)=512, Number of seeds=32026.06 | 20.3 | |
| KV-DistillK (Cache budget)=328, Number of seeds=32026.06 | 19.8 | |
| KV-DistillK (Cache budget)=1024, Number of seeds=32026.06 | 19.5 | |
| StillK (Cache budget)=82, Number of seeds=32026.06 | 18.9 | |
| KV-DistillK (Cache budget)=82, Number of seeds=32026.06 | 18.1 |