Long-context retrieval and reasoning on RULER (test)
92.72Retrieval (EM)Llama 3.1 8B Instruct
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Llama 3.1 8B InstructBase Model=Llama 3.1 8B Instruct, KV Cache Management Method=Full Attention, KV Cache Length/Budget=Full2025.11 | 92.72 | 68.08 | 36.07 | 59.13 | |
| SnapStreamBase Model=Llama 3.1 8B Instruct, KV Cache Management Method=SnapStream, KV Cache Length/Budget=8k2025.11 | 87.38 | 68.48 | 27.74 | 59.68 | |
| SAGE-KVBase Model=Llama 3.1 8B Instruct, KV Cache Management Method=SAGE-KV, KV Cache Length/Budget=8k2025.11 | 60.43 | 65.24 | 24.92 | 58.78 | |
| StreamingLLMBase Model=Llama 3.1 8B Instruct, KV Cache Management Method=StreamingLLM, KV Cache Length/Budget=8k2025.11 | 6.14 | 3.64 | 37.42 | 45.82 |