Long-context understanding on RULER 32K-HARD
88.74Average AccuracySDPA
Evaluation Results
| Method | Links | |
|---|---|---|
| SDPABackbone=Llama-3.1-8B-Inst, Sparsity=0%2025.10 | 88.74 | |
| vAttention(oracle-top-k)Backbone=Llama-3.1-8B-Inst, Sparsity=10%2025.10 | 88.61 | |
| oracle-top-kBackbone=Llama-3.1-8B-Inst, Sparsity=10%2025.10 | 87.18 | |
| vAttention(HAT)Backbone=Llama-3.1-8B-Inst, Sparsity=10%2025.10 | 86.56 | |
| HATBackbone=Llama-3.1-8B-Inst, Sparsity=10%2025.10 | 81.94 | |
| SDPABackbone=Dpsk-R1-Distill-Llama-8B, Sparsity=0%2025.10 | 65.41 | |
| vAttention(oracle-top-k)Backbone=Dpsk-R1-Distill-Llama-8B, Sparsity=10%2025.10 | 65.15 | |
| vAttention(HAT)Backbone=Dpsk-R1-Distill-Llama-8B, Sparsity=10%2025.10 | 65.06 | |
| oracle-top-kBackbone=Dpsk-R1-Distill-Llama-8B, Sparsity=10%2025.10 | 64.87 | |
| oracle-top-kBackbone=Mistral-7B-Inst-v0.3, Sparsity=10%2025.10 | 64.37 | |
| vAttention(oracle-top-k)Backbone=Mistral-7B-Inst-v0.3, Sparsity=10%2025.10 | 64.12 | |
| SDPABackbone=Mistral-7B-Inst-v0.3, Sparsity=0%2025.10 | 64.05 | |
| HATBackbone=Dpsk-R1-Distill-Llama-8B, Sparsity=10%2025.10 | 60.7 | |
| vAttention(HAT)Backbone=Mistral-7B-Inst-v0.3, Sparsity=10%2025.10 | 56.9 | |
| HATBackbone=Mistral-7B-Inst-v0.3, Sparsity=10%2025.10 | 54.66 |