Long-range memory evaluation on LongMemEval 500 samples 24k (test)
73AccuracyFull Attention
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Full AttentionFA Layers=36/36 (all), SWA window=2048, keep_first=10, FA decode=true2026.06 | 73 | 8.4 | |
| 1/2-FA PeriodicFA Layers=18/36 (every 2nd), SWA window=2048, keep_first=10, FA decode=true2026.06 | 65 | 0.4 | |
| NLL-Guided 1/4-FA (Ours)FA Layers=9/36 [1,3,9,10,12,13,15,21,34], SWA window=2048, keep_first=10, FA decode=true2026.06 | 64.6 | — | |
| 1/4-FA PeriodicFA Layers=9/36 [0,4,8,12,16,20,24,28,32], SWA window=2048, keep_first=10, FA decode=true2026.06 | 54.2 | 10.4 | |
| LightTransfer 1/4-FAFA Layers=9/36 [2,3,4,5,6,26,29,31,33], SWA window=2048, keep_first=10, FA decode=true2026.06 | 38.2 | 26.4 | |
| Naive SWAFA Layers=0/36 (none), SWA window=2048, keep_first=none, FA decode=false2026.06 | 3.2 | 61.4 |