Long-context understanding on RULER (test)
95.94RULER Accuracy (4K)Sketch&Walk
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Sketch&WalkModel=L3.1 8B, Phase=Decoding, Sparsity=80%2026.02 | 95.94 | 96.19 | 93.13 | 88.63 | 81.5 | 91.08 | |
| DenseModel=L3.1 8B, Phase=Decoding, Sparsity=0%2026.02 | 95.31 | 95.31 | 94.06 | 89.69 | 82.81 | 91.44 | |
| AdamasModel=L3.1 8B, Phase=Decoding, Sparsity=80%2026.02 | 95.31 | 94.06 | 94.06 | 89.06 | 81.19 | 90.74 | |
| QuestModel=L3.1 8B, Phase=Decoding, Sparsity=80%2026.02 | 91.88 | 92.81 | 92.81 | 87.5 | 80.31 | 89.06 |