Needle Retrieval on Needle In A Haystack
67Exact-match AccuracyKV-CAT
Evaluation results
| Method | Links | |
|---|---|---|
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=50%2026.05 | 67 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=40%2026.05 | 55 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=35%2026.05 | 54 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=50%2026.05 | 49 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=50%2026.05 | 47 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=35%2026.05 | 46 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=30%2026.05 | 44 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=40%2026.05 | 42 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=30%2026.05 | 41 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=40%2026.05 | 38 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=Mean2026.05 | 36.9 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=30%2026.05 | 34 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=35%2026.05 | 32 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=Mean2026.05 | 31.7 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=50%2026.05 | 28 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=25%2026.05 | 28 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=Mean2026.05 | 26 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=40%2026.05 | 24 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=25%2026.05 | 24 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=30%2026.05 | 23 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=20%2026.05 | 23 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=25%2026.05 | 22 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=20%2026.05 | 22 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=35%2026.05 | 21 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=15%2026.05 | 21 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=15%2026.05 | 21 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=25%2026.05 | 20 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=5%2026.05 | 20 | |
| QWEN2.5-1.5BTraining=Base, Keep Ratio=10%2026.05 | 20 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=5%2026.05 | 20 | |
| KV-CATBackbone=QWEN2.5-1.5B, Keep Ratio=10%2026.05 | 20 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=Mean2026.05 | 19.6 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=5%2026.05 | 18 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=5%2026.05 | 17 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=15%2026.05 | 16 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=15%2026.05 | 15 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=20%2026.05 | 15 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=10%2026.05 | 15 | |
| KV-CATBackbone=QWEN2.5-0.5B, Keep Ratio=20%2026.05 | 13 | |
| QWEN2.5-0.5BTraining=Base, Keep Ratio=10%2026.05 | 12 |