List your company
ResearchFormerly WizWand

Retrieval on Needle-in-A-Haystack RULER

100Success Rate (4K Context)FullAttn
34.89651.79868.785.602Nov 25, 2025
Updated 5mo ago

Evaluation results

MethodLinks
FullAttnTraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Full Attention, Receptive Field=Full2025.111001004835.470.9
MoBATraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Full Attention, Receptive Field=Full2025.1110032.42.2033.7
SSATraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Full Attention, Receptive Field=Full2025.111001008335.479.6
FullAttnTraining Protocol=Continual Trained to 32k, Inference Mode=Full Attention, Receptive Field=Full2025.1110010099.293.298.1
SSATraining Protocol=Continual Trained to 32k, Inference Mode=Full Attention, Receptive Field=Full2025.1110010099.870.692.6
MoBATraining Protocol=Continual Trained to 32k, Inference Mode=Full Attention, Receptive Field=Full2025.119973.25.41.244.7
SSATraining Protocol=Continual Trained to 32k, Inference Mode=Sparse Attention, Receptive Field=10242025.119455.617.25.443.1
MoBATraining Protocol=Continual Trained to 32k, Inference Mode=Sparse Attention, Receptive Field=10242025.1193.836.217.69.439.3
SSATraining Protocol=Continual Trained to 32k, Inference Mode=Sparse Attention, Receptive Field=2562025.1189.234.87.83.233.8
MoBATraining Protocol=Continual Trained to 32k, Inference Mode=Sparse Attention, Receptive Field=2562025.118227.612.4331.3
SSATraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Sparse Attention, Receptive Field=2562025.1181.633.65.82.230.8
SSATraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Sparse Attention, Receptive Field=10242025.1177.447.8—3.834.8
NSATraining Protocol=Continual Trained to 32k, Inference Mode=Sparse Attention, Receptive Field=10242025.1174.427.810.66.629.9
MoBATraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Sparse Attention, Receptive Field=2562025.1168.828.214.4027.9
NSATraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Sparse Attention, Receptive Field=10242025.1164.62511.4426.3
NSATraining Protocol=Continual Trained to 32k, Inference Mode=Sparse Attention, Receptive Field=2562025.116134.25.44.826.4
FullAttnTraining Protocol=Continual Trained to 32k, Inference Mode=Sparse Attention, Receptive Field=10242025.1159259.25.224.6
MoBATraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Sparse Attention, Receptive Field=10242025.1158.427.816827.6
FullAttnTraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Sparse Attention, Receptive Field=10242025.1158.221.47.24.422.8
FullAttnTraining Protocol=Continual Trained to 32k, Inference Mode=Sparse Attention, Receptive Field=2562025.115311.85.43.618.5
FullAttnTraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Sparse Attention, Receptive Field=2562025.11489.24.22.816.1
NSATraining Protocol=Length Extrapolation (RoPE Scaling), Inference Mode=Sparse Attention, Receptive Field=2562025.1137.413.83.62.414.3