Linguistic Minimal Pairs Evaluation on BLiMP
80.1AccuracyGPT-2
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-2Parameters=240M2026.05 | 80.1 | 9 | 6 | 5 | 2 | 2 | 4 | 1 | — | |
| QwenParameters=9B2026.05 | 79.1 | 6 | 1 | 3 | 1 | 0 | 2 | 0 | — | |
| QwenParameters=2B2026.05 | 75.8 | 10 | 0 | 4 | 0 | 0 | 2 | 0 | — | |
| GemmaParameters=270M2026.05 | 75.2 | 13 | 0 | 1 | 0 | 0 | 1 | 0 | — | |
| GemmaParameters=4B2026.05 | 74.6 | 8 | 4 | 1 | 3 | 0 | 1 | 0 | — | |
| GemmaParameters=1B2026.05 | 74.1 | 13 | 1 | 1 | 1 | 0 | 1 | 0 | — | |
| GELUEpoch=1, Number of layers=12L, Parameter-matched=true2026.06 | — | — | — | — | — | — | — | — | 80.7 | |
| GELUEpoch=2, Number of layers=12L, Parameter-matched=true2026.06 | — | — | — | — | — | — | — | — | 82.3 | |
| NC-FFNEpoch=1, Number of layers=12L, Parameter-matched=true2026.06 | — | — | — | — | — | — | — | — | 78.8 | |
| NC-FFNEpoch=2, Number of layers=12L, Parameter-matched=true2026.06 | — | — | — | — | — | — | — | — | 79 |