Zero-shot Language Modeling on CBT (test)
84.2AccuracySwitchHead MAC-matched
Evaluation results
| Method | Links | |
|---|---|---|
| SwitchHead MAC-matchedtotal params=376M2023.12 | 84.2 | |
| Transformertotal params=262M2023.12 | 83.6 | |
| SwitchHeadtotal params=262M2023.12 | 83.3 | |
| SwitchHead Shared selectiontotal params=262M2023.12 | 82.7 |