Language Modeling on QWEN3-like (train)
2.398LossMACRO-spec
Evaluation Results
| Method | Links | |
|---|---|---|
| MACRO-specParameter Scale=1B2026.05 | 2.398 | |
| MACRO-froParameter Scale=1B2026.05 | 2.403 | |
| MuonH-froParameter Scale=1B2026.05 | 2.404 | |
| MuonH-specParameter Scale=1B2026.05 | 2.409 | |
| MuonParameter Scale=1B2026.05 | 2.419 | |
| MACRO-specParameter Scale=330M2026.05 | 2.665 | |
| MACRO-froParameter Scale=330M2026.05 | 2.67 | |
| SSOParameter Scale=330M2026.05 | 2.675 | |
| MuonH-specParameter Scale=330M2026.05 | 2.679 | |
| MuonH-froParameter Scale=330M2026.05 | 2.68 | |
| MuonParameter Scale=330M2026.05 | 2.684 | |
| FSOParameter Scale=330M2026.05 | 2.69 | |
| FSOParameter Scale=120M2026.05 | 2.99 | |
| MACRO-froParameter Scale=120M2026.05 | 2.995 | |
| MuonH-froParameter Scale=120M2026.05 | 2.997 | |
| SSOParameter Scale=120M2026.05 | 3.001 | |
| MACRO-specParameter Scale=120M2026.05 | 3.007 | |
| MuonParameter Scale=120M2026.05 | 3.008 | |
| MuonH-specParameter Scale=120M2026.05 | 3.008 |