Overall Performance on Average (HellaSwag, PiQA, OBQA, COPA, LogiQA, WinoG, SciQ, ARC-E, Lambada)
42.5AccuracyDoGraph
Evaluation results
| Method | Links | |
|---|---|---|
| DoGraphBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 42.5 | |
| RegMixBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 42.1 | |
| DoReMiBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 41.7 | |
| UniformBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 40.9 | |
| Data Mixing LawBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 40.6 | |
| DOGEBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 40.5 | |
| Dynamic Loss-BasedBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 39.9 |