Language Modeling on WikiText-103 contaminated (test)
47.61Perplexity (PPL)Switch-ACMoE
Evaluation results
| Method | Links | |
|---|---|---|
| Switch-ACMoEBackbone=Switch2025.02 | 47.61 | |
| GLaM-ACMoEBackbone=GLaM2025.02 | 47.91 | |
| Switch TransformerBackbone=Switch2025.02 | 48.12 | |
| GLaMBackbone=GLaM2025.02 | 50.84 |