Pretraining Data Mixture Estimation on OLMo Pretraining Mixture (Temporal held-out)
95.5Web Source EstimateLLMSurgeon
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LLMSurgeonModel=OLMo-22026.05 | 95.5 | 2.1 | 10 | 1.5 | 0.5 | 0.3 | 89.18 | |
| LLMSurgeonModel=OLMo-1B2026.05 | 81.1 | 13.4 | 10 | 0.2 | 2.3 | 2.9 | 94.46 | |
| LLMSurgeonModel=OLMo-3 (held-out)2026.05 | 76.88 | 7.06 | 4 | 12.6 | 0.82 | 2.6 | 86.41 |