Grammaticality Judgment on BLiMP Epoch 2
82.3BLiMP Accuracy (Epoch 2)GELU baseline
Evaluation Results
| Method | Links | |
|---|---|---|
| GELU baselineLayers=12L, Parameter matching=true, Seed=Single2026.06 | 82.3 | |
| NC-FFN + decay + gateLayers=12L, Parameter matching=true, Seed=Single2026.06 | 81 | |
| NC-FFN + decay (learned γ, λ)Layers=12L, Parameter matching=true, Seed=Single2026.06 | 80 | |
| NC-FFN + quant (no forgetting)Layers=12L, Parameter matching=true, Seed=Single2026.06 | 79.3 | |
| NC-FFN ({∩, \})Layers=12L, Parameter matching=true, Seed=Single2026.06 | 79 |