Long-context retrieval on NIAH-Single-2 (128K context)
97AccuracyPre-train
Evaluation Results
| Method | Links | |
|---|---|---|
| Pre-trainModel=HypeNet-5B2026.06 | 97 | |
| +QK-RestoreModel=HypeNet-5B2026.06 | 96.8 | |
| +SFTModel=HypeNet-5B2026.06 | 91.4 | |
| Pre-trainModel=HypeNet-2B2026.06 | 88 | |
| +QK-RestoreModel=HypeNet-2B2026.06 | 83.2 | |
| Pre-trainModel=HypeNet-9B2026.06 | 72.4 | |
| Pre-trainModel=Jet-Nemotron-2B2026.06 | 61 | |
| +SFTModel=HypeNet-2B2026.06 | 48.4 | |
| +QK-RestoreModel=HypeNet-9B2026.06 | 44 | |
| +QK-RestoreModel=Jet-Nemotron-2B2026.06 | 22 | |
| +SFTModel=Jet-Nemotron-2B2026.06 | 21 | |
| +SFTModel=HypeNet-9B2026.06 | 17.4 |