Long-context Retrieval on S-NIAH (Exact Match)
99.6Exact Match AccuracyLlama3.1 8B Instruct
Evaluation results
| Method | Links | |
|---|---|---|
| Llama3.1 8B InstructAdaptation strategy=None2026.05 | 99.6 | |
| OffShelf-FT-SPEED+BoS-24Adaptation strategy=SPEED+BoS, K=242026.05 | 99.6 | |
| Llama3.1 8B Instruct2026.05 | 99.6 | |
| OffShelf-FT-SPEED+BoSSPEED cutoff (K)=242026.05 | 99.6 | |
| Full-depth LoRAAdaptation strategy=Full-depth LoRA2026.05 | 97.7 | |
| Full-depth LoRA2026.05 | 97.7 | |
| OffShelf-FT-SPEED+BoS-28Adaptation strategy=SPEED+BoS, K=282026.05 | 97 | |
| OffShelf-FT-SPEED+BoSSPEED cutoff (K)=282026.05 | 97 | |
| OffShelf-FT-SPEED+BoS-20Adaptation strategy=SPEED+BoS, K=202026.05 | 96.1 | |
| OffShelf-FT-SPEED+BoSSPEED cutoff (K)=202026.05 | 96.1 | |
| OffShelf-FT-SPEED+BoS-16Adaptation strategy=SPEED+BoS, K=162026.05 | 88.8 | |
| OffShelf-FT-SPEED+BoSSPEED cutoff (K)=162026.05 | 88.8 |