Zero-shot Reasoning on Gen5
59.7Accuracy (Zero-shot)Reg. MoE
Evaluation results
| Method | Links | |
|---|---|---|
| Reg. MoE# train tokens=1T, Active parameters=1B2026.05 | 59.7 | |
| EMO# train tokens=1T, Active parameters=1B2026.05 | 57.9 | |
| OLMoE# train tokens=5T, Active parameters=1B2026.05 | 57.6 | |
| Reg. MoE# train tokens=130B, Active parameters=1B2026.05 | 51 | |
| EMO# train tokens=130B, Active parameters=1B2026.05 | 49.2 | |
| Dense# train tokens=130B, Active parameters=1B2026.05 | 41.5 |