Long-context Retrieval on NIAH 128k
24.4Single ScoreOriginal
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OriginalModel Size=14B, Length Extrapolation=YaRN factor 42026.04 | 24.4 | 18.1 | 21.3 | |
| OriginalModel Size=8B, Length Extrapolation=YaRN factor 42026.04 | 24.3 | 14.7 | 19.5 | |
| OriginalModel Size=30B, Length Extrapolation=YaRN factor 42026.04 | 22.4 | 16.5 | 19.4 | |
| OriginalModel Size=1.7B, Length Extrapolation=YaRN factor 42026.04 | 21.6 | 6.1 | 13.8 | |
| BOSCHModel Size=14B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 18.7 | 6.5 | 12.6 | |
| FISHERModel Size=14B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 15.3 | 6.4 | 10.9 | |
| BOSCHModel Size=30B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 15.3 | 4.9 | 10.1 | |
| BOSCHModel Size=1.7B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 11.7 | 5.1 | 8.4 | |
| BOSCHModel Size=8B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 10 | 2.4 | 6.2 | |
| B-layerModel Size=8B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 9.9 | 1.2 | 5.6 | |
| FISHERModel Size=8B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 9.1 | 1.1 | 5.1 | |
| FISHERModel Size=30B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 8.8 | 3.1 | 6 | |
| B-layerModel Size=30B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 8.7 | 1.5 | 5.1 | |
| FISHERModel Size=1.7B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 6.9 | 1.8 | 4.4 | |
| INTRModel Size=30B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 6.1 | 1.5 | 3.8 | |
| B-layerModel Size=1.7B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 4.5 | 2.3 | 3.4 | |
| B-layerModel Size=14B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 2.2 | 1.1 | 1.7 | |
| INTRModel Size=8B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 1.7 | 1.1 | 1.4 | |
| INTRModel Size=1.7B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 1.5 | 1 | 1.3 | |
| INTRModel Size=14B, SWA ratio=0.75, Length Extrapolation=YaRN factor 42026.04 | 1.5 | 0.9 | 1.2 |