Language Modeling on EnWik-8 (test)
1.122BPCSmoothSMoE
Evaluation Results
| Method | Links | |
|---|---|---|
| SmoothSMoE2026.06 | 1.122 | |
| Switch-ACMoEPre-training dataset=EnWik-82025.02 | 1.137 | |
| Switch TransformerPre-training dataset=EnWik-82025.02 | 1.153 | |
| SMoE2026.06 | 1.153 |
Loading the SOTA2 catalog…