Long-context evaluation on RULER 64k (matched-train holdout)
39.9AccuracyStill
Evaluation results
| Method | Links | |
|---|---|---|
| StillK (Cache budget)=2048, Number of seeds=32026.06 | 39.9 | |
| StillK (Cache budget)=1024, Number of seeds=32026.06 | 36.5 | |
| StillK (Cache budget)=328, Number of seeds=32026.06 | 36.5 | |
| StillK (Cache budget)=164, Number of seeds=32026.06 | 35.1 | |
| StillK (Cache budget)=512, Number of seeds=32026.06 | 31.8 | |
| StillK (Cache budget)=82, Number of seeds=32026.06 | 24.2 | |
| KV-DistillK (Cache budget)=82, Number of seeds=32026.06 | 21.9 | |
| KV-DistillK (Cache budget)=512, Number of seeds=32026.06 | 21.1 | |
| KV-DistillK (Cache budget)=2048, Number of seeds=32026.06 | 21 | |
| KV-DistillK (Cache budget)=1024, Number of seeds=32026.06 | 20.4 | |
| KV-DistillK (Cache budget)=328, Number of seeds=32026.06 | 20.4 | |
| KV-DistillK (Cache budget)=164, Number of seeds=32026.06 | 20.2 |