Long-context understanding on RULER 128K
88.3AccuracyQwen3.5-35B-A3B-Base
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3.5-35B-A3B-BaseContext Window=128K, Protocol=Official execution protocol2026.04 | 88.3 | — | — | — | |
| XAttentionBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 88.13 | — | — | — | |
| DenseBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 87.81 | — | — | — | |
| CompactAttention-FPBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 87.03 | — | — | — | |
| FlashPrefillBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 86.77 | — | — | — | |
| QUOKABackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 79.03 | — | — | — | |
| JoyAI-LLM Flash-BaseContext Window=128K, Protocol=Official execution protocol2026.04 | 77 | — | — | — | |
| Dense FlashAttention-2Sparsity=All2025.12 | 76.79 | 1 | 1 | 1 | |
| DenseBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 76.59 | — | — | — | |
| BSFASparsity=k=10242025.12 | 76.54 | 0.76 | 0.74 | 1.06 | |
| BSFASparsity=k=7682025.12 | 75.75 | 0.61 | 0.61 | 1.11 | |
| SpargeAttentionSparsity=τ=0.72025.12 | 75.42 | — | — | 1.05 | |
| SpargeAttentionSparsity=τ=0.62025.12 | 75.28 | — | — | 1.09 | |
| XAttentionBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 74.5 | — | — | — | |
| CompactAttention-SABackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 74.28 | — | — | — | |
| CompactAttention-FPBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 74.17 | — | — | — | |
| FlashPrefillBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 74.12 | — | — | — | |
| FullKVBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 73.72 | — | — | — | |
| BSFASparsity=k=5122025.12 | 73.71 | 0.44 | 0.44 | 1.16 | |
| SeerAttentionBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 73.34 | — | — | — | |
| SpargeAttentionSparsity=τ=0.52025.12 | 70.68 | — | — | 1.17 | |
| QUOKABackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 70.44 | — | — | — | |
| Qwen3-30B-A3B-BaseContext Window=128K, Protocol=Official execution protocol2026.04 | 61.8 | — | — | — | |
| LOOKAHEADKVBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 54.83 | — | — | — | |
| SpecKVBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 52.62 | — | — | — | |
| LAQBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 50.67 | — | — | — | |
| SnapKVBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 30.56 | — | — | — |