List your company
ResearchFormerly WizWand

Needle In A Haystack on Ruler NIAH Single 3

84AccuracyBAM SSMax
0.822.44465.6May 28, 2025
Updated 8mo ago

Evaluation results

MethodLinks
BAM SSMaxPosition Encoding=BAM SSMax, Model Size=120M, Context Length=1K2025.0584
BAM SSMaxPosition Encoding=BAM SSMax, Model Size=120M, Context Length=1.5K2025.0568
BAM SSMaxPosition Encoding=BAM SSMax, Model Size=120M, Context Length=2K2025.0542
RoPE SSMaxPosition Encoding=RoPE SSMax, Model Size=120M, Context Length=1.5K2025.0530
RoPE SSMaxPosition Encoding=RoPE SSMax, Model Size=120M, Context Length=1K2025.0528
ALiBi SSMaxPosition Encoding=ALiBi SSMax, Model Size=120M, Context Length=1K2025.0524
RoPE SSMaxPosition Encoding=RoPE SSMax, Model Size=120M, Context Length=2K2025.0510
RoPE LocalPosition Encoding=RoPE Local, Model Size=120M, Context Length=1K2025.0510
Sinusoidal SSMaxPosition Encoding=Sinusoidal SSMax, Model Size=120M, Context Length=1K2025.058
BAM SSMaxPosition Encoding=BAM SSMax, Model Size=120M, Context Length=3K2025.058
RoPE SSMaxPosition Encoding=RoPE SSMax, Model Size=120M, Context Length=3K2025.056
NoPE SSMaxPosition Encoding=NoPE SSMax, Model Size=120M, Context Length=1K2025.054
ALiBi SSMaxPosition Encoding=ALiBi SSMax, Model Size=120M, Context Length=1.5K2025.054