Needle-In-A-Haystack Retrieval on NIAH
100NIAH ScoreTransformer
Evaluation results
| Method | Links | |
|---|---|---|
| Transformermodel size=152M, mb=42026.06 | 100 | |
| SISAmodel size=152M, ds=16, mb=42026.06 | 100 | |
| SISAmodel size=152M, ds=32, mb=22026.06 | 100 | |
| SISAmodel size=152M, ds=64, mb=42026.06 | 100 | |
| SISAmodel size=152M, ds=128, mb=42026.06 | 100 | |
| Mamba-3model size=152M, mb=42026.06 | 99 | |
| Mamba-2model size=152M, mb=42026.06 | 82.5 | |
| Full KVKV Compression=Disabled2026.05 | 37.51 | |
| AMS-TOVAKV Compression=Enabled2026.05 | 23.84 | |
| AMS-ExpectedKV Compression=Enabled2026.05 | 22.09 | |
| StreamingLLMKV Compression=Enabled2026.05 | 21.13 | |
| TOVAKV Compression=Enabled2026.05 | 18.94 | |
| PyramidKVKV Compression=Enabled2026.05 | 17.77 | |
| AdaKV-ExpE2KV Compression=Enabled2026.05 | 8.66 |