Code on HumanEval (test)
58.1HumanEval Success RateLightMoE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LightMoERatio=30%, # Params=0.96B2026.03 | 58.1 | 51.4 | |
| LoRARatio=0%, # Params=0.96B2026.03 | 58 | 52 | |
| LightMoERatio=40%, # Params=0.96B2026.03 | 55.8 | 50.9 | |
| OriginalRatio=0%, # Params=-2026.03 | 53.9 | 26.2 | |
| MC-SMoERatio=30%, # Params=0.96B2026.03 | 53.7 | 49.3 | |
| HC-SMoERatio=30%, # Params=0.96B2026.03 | 51.6 | 49.8 | |
| LightMoERatio=50%, # Params=0.96B2026.03 | 51.5 | 48.4 | |
| MC-SMoERatio=40%, # Params=0.96B2026.03 | 50.2 | 48.4 | |
| HC-SMoERatio=40%, # Params=0.96B2026.03 | 47.8 | 48.9 | |
| MoBERatio=30%, # Params=0.96B2026.03 | 47.7 | 53.8 | |
| HC-SMoERatio=50%, # Params=0.96B2026.03 | 43.5 | 45.8 | |
| MoBERatio=40%, # Params=0.96B2026.03 | 40.6 | 49 | |
| MC-SMoERatio=50%, # Params=0.96B2026.03 | 38.2 | 40.4 | |
| Dense baselineModel Type=Dense, Compute=5.45e212025.06 | 22.56 | — | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=strict2025.06 | 21.95 | — | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=-2025.06 | 21.34 | — | |
| MoBERatio=50%, # Params=0.96B2026.03 | 19.2 | 38.6 |