Hyper-parameter Search on MoE-3B
3.03FLOPsGrid Search
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Grid SearchStrategy=Grid Search, Proxy Models=9 × MoE-3B2026.06 | 3.03 | — | |
| Ours (Scaling)Strategy=Ours (Scaling), Proxy Models=9 × {100M, 500M}2026.06 | 8.15 | 73.1 |
Loading the SOTA2 catalog…