Scaling Law Fitting on DCLM strongly regularized baseline 20 points (Full fit)
0.008RMSESoftQ
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SoftQ# params=52026.06 | 0.008 | 0.0052 | -183.06 | |
| Quanta# params=42026.06 | 0.0125 | 0.0089 | -167.23 | |
| Muennighoff# params=72026.06 | 0.0233 | 0.0171 | -136.29 | |
| Chinchilla# params=52026.06 | 0.0265 | 0.018 | -135.18 |