Zero-shot Language Modeling on BLiMP (test)
79.6AccuracySwitchHead
Evaluation results
| Method | Links | |
|---|---|---|
| SwitchHeadtotal params=262M2023.12 | 79.6 | |
| SwitchHead MAC-matchedtotal params=376M2023.12 | 79.4 | |
| SwitchHead Shared selectiontotal params=262M2023.12 | 79.4 | |
| SwitchHead MAC-matchedtotal params=63M2023.12 | 77.1 | |
| Transformertotal params=262M2023.12 | 76.1 | |
| SwitchHeadtotal params=47M2023.12 | 75.7 | |
| SwitchHead Shared selectiontotal params=47M2023.12 | 74.6 | |
| Transformertotal params=47M2023.12 | 73.6 |