Pre-training on FineWeb 124M Transformer (val)
2.1Training Time to Loss 3.28 (min)Best Human Expert
Evaluation results
| Method | Links | |
|---|---|---|
| Best Human Expertparameters=124M, target_loss=3.282026.01 | 2.1 | |
| Execution-Guided Searchparameters=124M, target_loss=3.282026.01 | 19.7 | |
| Baselinemodel_type=nanoGPT, parameters=124M, target_loss=3.282026.01 | 35.9 |