Long-context understanding on RULER 64K
92.37AccuracyRetroInfer
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| RetroInferModel=Qwen2.5-14B2025.11 | 92.37 | — | — | — | |
| LiteCache + HATAModel=Qwen2.5-14B2025.11 | 92.35 | — | — | — | |
| Qwen2.5-14BModel=Qwen2.5-14B2025.11 | 92.29 | — | — | — | |
| N-3-Super 120B-A12B-BaseShots=02026.04 | 92.26 | — | — | — | |
| HATA AlgoModel=Qwen2.5-14B2025.11 | 92.24 | — | — | — | |
| DenseBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 90.56 | — | — | — | |
| InfiniGenModel=Qwen2.5-14B2025.11 | 89.74 | — | — | — | |
| XAttentionBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 88.92 | — | — | — | |
| CompactAttention-FPBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 88.43 | — | — | — | |
| FlashPrefillBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 87.73 | — | — | — | |
| Dense FlashAttention-2Sparsity=All2025.12 | 84.96 | 100 | 100 | 1 | |
| BSFASparsity=k=3842025.12 | 84.24 | 62 | 61 | 1.05 | |
| FullKVBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 84.15 | — | — | — | |
| CompactAttention-SABackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 84.12 | — | — | — | |
| CompactAttention-FPBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 83.96 | — | — | — | |
| FlashPrefillBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 83.94 | — | — | — | |
| SeerAttentionBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 83.82 | — | — | — | |
| DenseBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 83.52 | — | — | — | |
| BSFASparsity=k=2562025.12 | 83.39 | 44 | 45 | 1.09 | |
| XAttentionBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 83.27 | — | — | — | |
| BSFASparsity=k=1922025.12 | 83.08 | 35 | 36 | 1.13 | |
| SpargeAttentionSparsity=τ=0.52025.12 | 83.03 | — | — | 1.03 | |
| GLM-4.5 Air-BaseShots=02026.04 | 80.26 | — | — | — | |
| QUOKABackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.05 | 79.99 | — | — | — | |
| QUOKABackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.05 | 79.57 | — | — | — | |
| HATA AlgoModel=Llama3-8B2025.11 | 76.28 | — | — | — | |
| Llama3-8BModel=Llama3-8B2025.11 | 76.26 | — | — | — | |
| RetroInferModel=Llama3-8B2025.11 | 76.26 | — | — | — | |
| LiteCache + HATAModel=Llama3-8B2025.11 | 75.94 | — | — | — | |
| SpargeAttentionSparsity=τ=0.42025.12 | 75.07 | — | — | 1.1 | |
| InfiniGenModel=Llama3-8B2025.11 | 72.96 | — | — | — | |
| Ling-flash base-2.0Shots=02026.04 | 72.12 | — | — | — | |
| LOOKAHEADKVBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 69.45 | — | — | — | |
| SpargeAttentionSparsity=τ=0.32025.12 | 66.04 | — | — | 1.19 | |
| LAQBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 64.1 | — | — | — | |
| SpecKVBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 64.02 | — | — | — | |
| SnapKVBackbone=LLaMA3.1-8B-Instruct, Cache Budget (C)=128, Sample size per task=502026.03 | 39.64 | — | — | — |