General Evaluation on Average Downstream Benchmark Suite
37.9Average AccuracyDoGraph
Evaluation results
| Method | Links | |
|---|---|---|
| DoGraphBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 37.9 | |
| Dynamic Loss-BasedBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 37.2 | |
| DoReMiBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 36.5 | |
| DOGEBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 36.5 | |
| RegMixBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 35.7 | |
| UniformBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 35.4 | |
| Data Mixing LawBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 35.4 |