Loss Curve Fitting across Model Sizes on Dense models
0.402ASMT (MAPE)ASMT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ASMTLRmax=1.25e-4, Batch Size=5122025.12 | 0.402 | — | — | |
| ASMTLRmax=5e-5, Batch Size=10242025.12 | 0.565 | — | — | |
| ASMTLRmax=5e-5, Batch Size=5122025.12 | 0.579 | — | — | |
| CMMT (λ = 0.99)LRmax=1.25e-4, Batch Size=5122025.12 | — | — | 0.403 | |
| CMMT (λ = 0.99)LRmax=5e-5, Batch Size=5122025.12 | — | — | 0.887 | |
| CMMT (λ = 0.99)LRmax=5e-5, Batch Size=10242025.12 | — | — | 0.567 | |
| CMMT (λ = 0.999)LRmax=1.25e-4, Batch Size=5122025.12 | — | 0.413 | — | |
| CMMT (λ = 0.999)LRmax=5e-5, Batch Size=5122025.12 | — | 0.943 | — | |
| CMMT (λ = 0.999)LRmax=5e-5, Batch Size=10242025.12 | — | 0.566 | — |