Needle-style retrieval on RULER-32K (test)
100SG-1 ScoreFull attention
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Full attentionBudget (KV tokens)=–, Training-free evaluation=true2026.05 | 100 | 99.6 | 99.6 | 99.4 | 100 | 99.8 | 98.55 | 99.05 | 99.96 | 88.46 | 93.27 | 75.8 | 57.4 | 93.15 | |
| QuestBudget (KV tokens)=2048, Training-free evaluation=true2026.05 | 100 | 98.8 | 97 | 99.2 | 99.8 | 97.6 | 99.45 | 99.3 | 99.88 | 77.8 | 90.67 | 75 | 57.2 | 91.67 | |
| H2OBudget (KV tokens)=2048, Training-free evaluation=true2026.05 | 100 | 99.8 | 31 | 99.6 | 97.8 | 1.8 | 92.65 | 65.2 | 98.4 | 42.96 | 90.8 | 74.4 | 56.8 | 73.17 | |
| InfLLMBudget (KV tokens)=2048, Training-free evaluation=true2026.05 | 100 | 94 | 85.2 | 95.2 | 99.8 | 96.4 | 93 | 88.45 | 94.08 | 22.1 | 83.8 | 74.6 | 57.2 | 83.37 | |
| UNIQUEBudget (KV tokens)=2048, Training-free evaluation=true2026.05 | 100 | 99.8 | 99.2 | 99.6 | 100 | 99.8 | 99 | 99.15 | 99.96 | 82.28 | 91.07 | 75.6 | 57.4 | 92.53 | |
| UNIQUEBudget (KV tokens)=512, Training-free evaluation=true2026.05 | 100 | 99.8 | 99.2 | 99.8 | 100 | 99.6 | 99.55 | 99.3 | 99.96 | 62.9 | 88.67 | 73.6 | 57.8 | 90.78 | |
| H2OBudget (KV tokens)=512, Training-free evaluation=true2026.05 | 99.8 | 98 | 2.6 | 97.6 | 84.6 | 0 | 44.45 | 14.65 | 66.88 | 20.94 | 68.33 | 74.4 | 55.8 | 56 | |
| QuestBudget (KV tokens)=512, Training-free evaluation=true2026.05 | 99.4 | 94.6 | 91 | 97.2 | 95.8 | 53 | 96.55 | 96.65 | 99 | 53.24 | 87.6 | 74 | 55.8 | 84.14 | |
| InfLLMBudget (KV tokens)=512, Training-free evaluation=true2026.05 | 92.6 | 86 | 76 | 88.6 | 98.4 | 43.8 | 85.6 | 79.4 | 87.52 | 10.12 | 74.4 | 74.2 | 56.2 | 73.3 |