Language Modeling on C5
30.39PerplexityAIR-MoE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AIR-MoESize=Medium2026.05 | 30.39 | 4.1 | |
| PEERSize=Medium2026.05 | 31.6 | 4.1 | |
| Coarse MoESize=Medium2026.05 | 32.1 | 4 | |
| Hierarchical MoESize=Medium2026.05 | 33.08 | 4.1 | |
| AIR-MoESize=Large2026.05 | 41.25 | 14.3 | |
| Coarse MoESize=Large2026.05 | 43.13 | 13.2 | |
| PEERSize=Large2026.05 | 45.37 | 14 | |
| Hierarchical MoESize=Large2026.05 | 45.65 | 14.4 | |
| AIR-MoESize=Small2026.05 | 131.81 | 625.5 | |
| Standard Granular MoESize=Small2026.05 | 132.34 | 817.2 | |
| PEERSize=Small2026.05 | 145.32 | 626.3 | |
| Hierarchical MoESize=Small2026.05 | 147.99 | 622.6 | |
| Coarse MoESize=Small2026.05 | 151.99 | 611.7 |