List your company
ResearchFormerly WizWand

Long-context understanding on RULER 32K

94.48AccuracyQwen2.5-14B
54.765.027575.35585.6825Nov 18, 2025Dec 17, 2025Jan 16, 2026Feb 15, 2026Mar 17, 2026Apr 16, 2026May 16, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Qwen2.5-14BModel=Qwen2.5-14B2025.1194.48
RetroInferModel=Qwen2.5-14B2025.1194.41
LiteCache + HATAModel=Qwen2.5-14B2025.1194.22
HATA AlgoModel=Qwen2.5-14B2025.1194.13
DenseBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.0593.01
CompactAttention-FPBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.0592.41
Dense AttentionModel=Llama-3.1-8B, Sparsity=Dense, Deployment Phase=Prefill Phase2025.1292.33
Dense AttentionModel=Llama-3.1-8B, Sparsity=Dense, Deployment Phase=Prefill + Decode Phase2025.1292.33
XAttentionBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.0592.29
InfiniGenModel=Qwen2.5-14B2025.1192.19
BLASSTModel=Qwen3-8B, Sparsity=~75%, Deployment Phase=Prefill Phase2025.1292.11
BLASSTModel=Qwen3-8B, Sparsity=~50%, Deployment Phase=Prefill Phase2025.1292.08
BLASSTModel=Qwen3-8B, Sparsity=~50%, Deployment Phase=Prefill + Decode Phase2025.1292.07
Dense AttentionBackbone=Qwen3-8B, Attention Method=Dense, Phase=Decode2025.1291.9
Dense AttentionModel=Qwen3-8B, Sparsity=Dense, Deployment Phase=Prefill Phase2025.1291.9
Dense AttentionModel=Qwen3-8B, Sparsity=Dense, Deployment Phase=Prefill + Decode Phase2025.1291.9
BLASSTModel=Llama-3.1-8B, Sparsity=~50%, Deployment Phase=Prefill Phase2025.1291.81
BLASSTModel=Llama-3.1-8B, Sparsity=~50%, Deployment Phase=Prefill + Decode Phase2025.1291.79
BLASSTModel=Qwen3-8B, Sparsity=~75%, Deployment Phase=Prefill + Decode Phase2025.1291.74
BLASSTModel=Llama-3.1-8B, Sparsity=~75%, Deployment Phase=Prefill Phase2025.1291.67
BLASSTModel=Llama-3.1-8B, Sparsity=~75%, Deployment Phase=Prefill + Decode Phase2025.1291.67
BLASSTBackbone=Qwen3-8B, Attention Method=BLASST, Sparsity=~50%, Phase=Decode2025.1291.55
FlashPrefillBackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.0591.55
SeerAttentionBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.0590.23
XAttentionBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.0589.16
FlashPrefillBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.0588.99
CompactAttention-SABackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.0588.92
CompactAttention-FPBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.0588.77
DenseBackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.0588.23
RocketKVBackbone=Qwen3-8B, Attention Method=RocketKV, Phase=Decode2025.1287.89
QUOKABackbone=LLaMA-3.1-8B-Instruct, Chunk Size=10242026.0583.52
QUOKABackbone=Qwen3-30B-A3B-Instruct-2507, Chunk Size=10242026.0581.96
HATA AlgoModel=Llama3-8B2025.1180.94
Llama3-8BModel=Llama3-8B2025.1180.8
LiteCache + HATAModel=Llama3-8B2025.1180.7
RetroInferModel=Llama3-8B2025.1180.64
InfiniGenModel=Llama3-8B2025.1176.76
QuestBackbone=Qwen3-8B, Attention Method=Quest, Phase=Decode2025.1256.23