Long-context language understanding on RULER 32K
85.8Easy Average ScoreHashAttention
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HashAttentionModel=Meta-llama/Llama-3.1-8B-Instruct, Context Window=32K, Sparsity=10%2025.10 | 85.8 | 81.94 | 83.87 | |
| oracle-top-kModel=Meta-llama/Llama-3.1-8B-Instruct, Context Window=32K, Sparsity=10%2025.10 | 85.31 | 87.18 | 86.25 | |
| full attentionModel=Meta-llama/Llama-3.1-8B-Instruct, Context Window=32K2025.10 | 85.3 | 88.74 | 87.02 | |
| vAttention(oracle-top-k)Model=Meta-llama/Llama-3.1-8B-Instruct, Context Window=32K, Sparsity=10%, Approximation=oracle-top-k2025.10 | 85.17 | 88.61 | 86.89 | |
| vAttention(HashAttention)Model=Meta-llama/Llama-3.1-8B-Instruct, Context Window=32K, Sparsity=10%, Approximation=HashAttention2025.10 | 84.57 | 86.56 | 85.57 |