List your company
ResearchFormerly WizWand

Long-context Retrieval on NIAH 64k

49.3Single ScoreOriginal
0.73213.34125.9538.559Apr 7, 2026
Updated 6mo ago

Evaluation results

MethodLinks
OriginalModel Size=8B, Length Extrapolation=YaRN factor 22026.0449.344.747
OriginalModel Size=30B, Length Extrapolation=YaRN factor 22026.0449.345.747.5
OriginalModel Size=14B, Length Extrapolation=YaRN factor 22026.0449.145.947.5
OriginalModel Size=1.7B, Length Extrapolation=YaRN factor 22026.0447.923.535.7
BOSCHModel Size=14B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0443.723.633.7
BOSCHModel Size=8B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0441.313.927.6
BOSCHModel Size=30B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0438.511.124.8
FISHERModel Size=8B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.04378.722.9
FISHERModel Size=14B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.043713.325.2
BOSCHModel Size=1.7B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.04321523.5
FISHERModel Size=1.7B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0427.77.517.6
FISHERModel Size=30B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0423.95.714.8
B-layerModel Size=1.7B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0423.78.216
B-layerModel Size=30B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0419.53.811.7
B-layerModel Size=8B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0418.11.69.9
INTRModel Size=30B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0417.32.59.9
B-layerModel Size=14B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.0412.12.77.4
INTRModel Size=8B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.042.91.82.4
INTRModel Size=1.7B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.042.61.72.2
INTRModel Size=14B, SWA ratio=0.75, Length Extrapolation=YaRN factor 22026.042.61.32