Long-context evaluation on RULER 32k (Zero-shot transfer)
37.2AccuracyStill
Evaluation results
| Method | Links | |
|---|---|---|
| StillK (Cache budget)=2048, Number of seeds=32026.06 | 37.2 | |
| StillK (Cache budget)=1024, Number of seeds=32026.06 | 34 | |
| StillK (Cache budget)=328, Number of seeds=32026.06 | 33.7 | |
| StillK (Cache budget)=164, Number of seeds=32026.06 | 33.3 | |
| StillK (Cache budget)=512, Number of seeds=32026.06 | 29.6 | |
| KV-DistillK (Cache budget)=2048, Number of seeds=32026.06 | 24.1 | |
| KV-DistillK (Cache budget)=1024, Number of seeds=32026.06 | 22.7 | |
| KV-DistillK (Cache budget)=164, Number of seeds=32026.06 | 22.7 | |
| StillK (Cache budget)=82, Number of seeds=32026.06 | 22.1 | |
| KV-DistillK (Cache budget)=328, Number of seeds=32026.06 | 22 | |
| KV-DistillK (Cache budget)=512, Number of seeds=32026.06 | 21.3 | |
| KV-DistillK (Cache budget)=82, Number of seeds=32026.06 | 18.8 |