Retrieval on Needle-in-a-Haystack L=8k
100AccuracyBaseline SFT
Evaluation results
| Method | Links | |
|---|---|---|
| Baseline SFT2026.04 | 100 | |
| MIN0.1Anchor=MIN, p=0.12026.04 | 100 | |
| MIN0.2Anchor=MIN, p=0.22026.04 | 100 | |
| MEAN0.1Anchor=MEAN, p=0.12026.04 | 100 | |
| MEAN0.2Anchor=MEAN, p=0.22026.04 | 100 | |
| Full ModelKV Cache Budget=Full2026.03 | 97.56 | |
| SemantiCacheKV Cache Budget=40962026.03 | 94.38 | |
| D2OKV Cache Budget=40962026.03 | 93.21 | |
| SemantiCacheKV Cache Budget=10242026.03 | 91.02 | |
| D2OKV Cache Budget=10242026.03 | 90.38 | |
| CaMKV Cache Budget=40962026.03 | 86.52 | |
| SnapKVKV Cache Budget=40962026.03 | 85.92 | |
| SnapKVKV Cache Budget=10242026.03 | 82.67 | |
| CaMKV Cache Budget=10242026.03 | 81.78 | |
| H2OKV Cache Budget=40962026.03 | 81.62 | |
| H2OKV Cache Budget=10242026.03 | 78.96 | |
| StreamingLLMKV Cache Budget=40962026.03 | 61.63 | |
| StreamingLLMKV Cache Budget=10242026.03 | 57.42 | |
| fp16bits=16.00, evaluation protocol=single-window2026.05 | 40 | |
| hqmq_s192_r4bits=4.04, evaluation protocol=single-window2026.05 | 40 | |
| naive int4bits=4.00, evaluation protocol=single-window2026.05 | 40 | |
| hqmq_s48_r4bits=3.54, evaluation protocol=single-window2026.05 | 40 | |
| hqmq_s24_r3bits=3.04, evaluation protocol=single-window2026.05 | 40 | |
| naive int3bits=3.00, evaluation protocol=single-window2026.05 | 32 |