List your company

Long-context understanding on RULER 32K-HARD

88.74Average AccuracySDPA
53.296862.498471.780.9016Oct 7, 2025

Evaluation results

MethodLinks
SDPABackbone=Llama-3.1-8B-Inst, Sparsity=0%2025.1088.74
vAttention(oracle-top-k)Backbone=Llama-3.1-8B-Inst, Sparsity=10%2025.1088.61
oracle-top-kBackbone=Llama-3.1-8B-Inst, Sparsity=10%2025.1087.18
vAttention(HAT)Backbone=Llama-3.1-8B-Inst, Sparsity=10%2025.1086.56
HATBackbone=Llama-3.1-8B-Inst, Sparsity=10%2025.1081.94
SDPABackbone=Dpsk-R1-Distill-Llama-8B, Sparsity=0%2025.1065.41
vAttention(oracle-top-k)Backbone=Dpsk-R1-Distill-Llama-8B, Sparsity=10%2025.1065.15
vAttention(HAT)Backbone=Dpsk-R1-Distill-Llama-8B, Sparsity=10%2025.1065.06
oracle-top-kBackbone=Dpsk-R1-Distill-Llama-8B, Sparsity=10%2025.1064.87
oracle-top-kBackbone=Mistral-7B-Inst-v0.3, Sparsity=10%2025.1064.37
vAttention(oracle-top-k)Backbone=Mistral-7B-Inst-v0.3, Sparsity=10%2025.1064.12
SDPABackbone=Mistral-7B-Inst-v0.3, Sparsity=0%2025.1064.05
HATBackbone=Dpsk-R1-Distill-Llama-8B, Sparsity=10%2025.1060.7
vAttention(HAT)Backbone=Mistral-7B-Inst-v0.3, Sparsity=10%2025.1056.9
HATBackbone=Mistral-7B-Inst-v0.3, Sparsity=10%2025.1054.66