Loss Curve Fitting across Batch Sizes on Model loss data (train)
0.529ASMT ScoreASMT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| ASMTModel=Dense, Model Size=100M2025.12 | 0.529 | — | — | |
| ASMTModel=Dense, Model Size=50M2025.12 | 0.544 | — | — | |
| ASMTModel=MoE, Model Size=1.5B2025.12 | 0.607 | — | — | |
| ASMTModel=Dense, Model Size=500M2025.12 | 0.646 | — | — | |
| ASMTModel=Dense, Model Size=1B2025.12 | 0.674 | — | — | |
| ASMTModel=MoE, Model Size=100M2025.12 | 1.223 | — | — | |
| ASMTModel=MoE, Model Size=700M2025.12 | 1.561 | — | — | |
| CMMT (λ = 0.99)Model=Dense, Model Size=50M2025.12 | — | — | 0.552 | |
| CMMT (λ = 0.99)Model=Dense, Model Size=100M2025.12 | — | — | 0.538 | |
| CMMT (λ = 0.99)Model=Dense, Model Size=500M2025.12 | — | — | 0.66 | |
| CMMT (λ = 0.99)Model=Dense, Model Size=1B2025.12 | — | — | 0.681 | |
| CMMT (λ = 0.99)Model=MoE, Model Size=100M2025.12 | — | — | 1.342 | |
| CMMT (λ = 0.99)Model=MoE, Model Size=700M2025.12 | — | — | 1.752 | |
| CMMT (λ = 0.99)Model=MoE, Model Size=1.5B2025.12 | — | — | 0.736 | |
| CMMT (λ = 0.999)Model=Dense, Model Size=50M2025.12 | — | 1.293 | — | |
| CMMT (λ = 0.999)Model=Dense, Model Size=100M2025.12 | — | 1.487 | — | |
| CMMT (λ = 0.999)Model=Dense, Model Size=500M2025.12 | — | 1.966 | — | |
| CMMT (λ = 0.999)Model=Dense, Model Size=1B2025.12 | — | 2.151 | — | |
| CMMT (λ = 0.999)Model=MoE, Model Size=100M2025.12 | — | 1.467 | — | |
| CMMT (λ = 0.999)Model=MoE, Model Size=700M2025.12 | — | 1.896 | — | |
| CMMT (λ = 0.999)Model=MoE, Model Size=1.5B2025.12 | — | 1.079 | — |