Knowledge Incorporation on SQuAD Continued Pretraining n=200
46.2AccuracySleep (Transformer + four-level)
Evaluation results
| Method | Links | |
|---|---|---|
| Sleep (Transformer + four-level)Memory system=Four-level2026.06 | 46.2 | |
| removing gradient-based selectionAblation=Gradient-based selection removed2026.06 | 45.2 | |
| removing random expertAblation=Random expert removed2026.06 | 44.7 | |
| Sleep (Transformer)Memory system=Two-level2026.06 | 44.3 | |
| SEAL2026.06 | 43.2 | |
| removing DreamingAblation=Dreaming phase removed2026.06 | 36.2 | |
| Fine-tuned Model with No Dreaming2026.06 | 32 | |
| Base model2026.06 | 31.9 |