Reading Comprehension on ARC-E
41.7AccuracyRegMix
Evaluation Results
| Method | Links | |
|---|---|---|
| RegMixBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 41.7 | |
| DoReMiBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 40.6 | |
| UniformBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 40 | |
| DOGEBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 40 | |
| DoGraphBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 39.2 | |
| Data Mixing LawBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 39 | |
| Dynamic Loss-BasedBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.04 | 38.9 | |
| DoGraphBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 33.9 | |
| DoReMiBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 32.3 | |
| DoGraphModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 32.3 | |
| Data Mixing LawBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 32 | |
| Dynamic Loss-BasedModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 31.8 | |
| Dynamic Loss-BasedBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 31.7 | |
| RegMixBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 31.7 | |
| UniformBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 31.4 | |
| DOGEBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.04 | 31.3 | |
| DoReMiModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 30.9 | |
| DOGEModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 30.8 | |
| UniformModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 30.6 | |
| Data Mixing LawModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 30.1 | |
| RegMixModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 29.8 |