Long-context understanding on RULER 32k tokens
100NIAH-SFlashAttn-2
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| FlashAttn-2Category=Dense, Model=Llama-3.1-8B-Instruct, KV retention=Dense (Full)2026.05 | 100 | 96 | 84 | 93.3 | |
| H2OCategory=Eviction, Model=Llama-3.1-8B-Instruct, KV retention=10%2026.05 | 100 | 96 | 52 | 82.7 | |
| QuestCategory=Retrieval, Model=Llama-3.1-8B-Instruct, KV retention=10%2026.05 | 100 | 96 | 60 | 85.3 | |
| LOUVERCategory=Ours, Model=Llama-3.1-8B-Instruct, KV retention=10%2026.05 | 100 | 98 | 74 | 90.7 | |
| StreamingLLMCategory=Eviction, Model=Llama-3.1-8B-Instruct, KV retention=10%2026.05 | 0 | 9.3 | 0 | 3.1 |