Needle-in-a-haystack retrieval on NIAH 64K 60 items, 3 needle positions (test)
28.2F1 ScoreFull cache
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Full cacheC=65536, Ret%=100%, model=Qwen3-4B-Instruct-25072026.05 | 28.2 | 29.3 | 27.5 | 27.8 | 100 | |
| LRU + protC=4096, Ret%=6.3%, model=Qwen3-4B-Instruct-25072026.05 | 11.2 | 29.7 | 2.4 | 1.5 | 39.8 | |
| LRU (no prot)C=4096, Ret%=6.3%, model=Qwen3-4B-Instruct-25072026.05 | 9.8 | 28.6 | 0 | 0.7 | 34.7 | |
| Random + protC=8192, Ret%=12.5%, model=Qwen3-4B-Instruct-25072026.05 | 4.2 | 2.8 | 5.8 | 4 | 14.9 | |
| H2O + protC=8192, Ret%=12.5%, model=Qwen3-4B-Instruct-25072026.05 | 3.5 | 5.6 | 3.5 | 1.4 | 12.4 | |
| Random + protC=4096, Ret%=6.3%, model=Qwen3-4B-Instruct-25072026.05 | 3 | 0.7 | 5.1 | 3.4 | 10.6 | |
| H2O + protC=4096, Ret%=6.3%, model=Qwen3-4B-Instruct-25072026.05 | 2.2 | 3.4 | 2.4 | 0.9 | 7.8 | |
| H2O (no prot)C=4096, Ret%=6.3%, model=Qwen3-4B-Instruct-25072026.05 | 0 | 0 | 0 | 0 | 0 |