List your company
ResearchFormerly WizWand

Loss Curve Fitting across Batch Sizes on Model loss data (train)

0.529ASMT ScoreASMT
0.487720.766361.0451.32364Dec 5, 2025
Updated 8mo ago

Evaluation results

MethodLinks
ASMTModel=Dense, Model Size=100M2025.120.529——
ASMTModel=Dense, Model Size=50M2025.120.544——
ASMTModel=MoE, Model Size=1.5B2025.120.607——
ASMTModel=Dense, Model Size=500M2025.120.646——
ASMTModel=Dense, Model Size=1B2025.120.674——
ASMTModel=MoE, Model Size=100M2025.121.223——
ASMTModel=MoE, Model Size=700M2025.121.561——
CMMT (λ = 0.99)Model=Dense, Model Size=50M2025.12——0.552
CMMT (λ = 0.99)Model=Dense, Model Size=100M2025.12——0.538
CMMT (λ = 0.99)Model=Dense, Model Size=500M2025.12——0.66
CMMT (λ = 0.99)Model=Dense, Model Size=1B2025.12——0.681
CMMT (λ = 0.99)Model=MoE, Model Size=100M2025.12——1.342
CMMT (λ = 0.99)Model=MoE, Model Size=700M2025.12——1.752
CMMT (λ = 0.99)Model=MoE, Model Size=1.5B2025.12——0.736
CMMT (λ = 0.999)Model=Dense, Model Size=50M2025.12—1.293—
CMMT (λ = 0.999)Model=Dense, Model Size=100M2025.12—1.487—
CMMT (λ = 0.999)Model=Dense, Model Size=500M2025.12—1.966—
CMMT (λ = 0.999)Model=Dense, Model Size=1B2025.12—2.151—
CMMT (λ = 0.999)Model=MoE, Model Size=100M2025.12—1.467—
CMMT (λ = 0.999)Model=MoE, Model Size=700M2025.12—1.896—
CMMT (λ = 0.999)Model=MoE, Model Size=1.5B2025.12—1.079—