Knowledge on MMLU (test)
32.92Accuracy (Knowledge, test)MoE w/ optimal AR
Evaluation results
| Method | Links | |
|---|---|---|
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=-2025.06 | 32.92 | |
| Dense baselineModel Type=Dense, Compute=5.45e212025.06 | 31.26 | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=strict2025.06 | 24.57 |