Language Modeling on The Pile (eval)
14.1Perplexity (PPL)Baseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineModel Size=1B2025.09 | 14.1 | |
| InductionModel Size=1B2025.09 | 14.9 | |
| Anti-inductionModel Size=1B2025.09 | 14.9 | |
| BalancedModel Size=1B2025.09 | 14.9 | |
| BaselineModel Size=0.5B2025.09 | 16 | |
| InductionModel Size=0.5B2025.09 | 17.9 | |
| Anti-inductionModel Size=0.5B2025.09 | 18.2 | |
| BalancedModel Size=0.5B2025.09 | 18.2 | |
| BaselineModel Size=0.13B2025.09 | 21.8 | |
| InductionModel Size=0.13B2025.09 | 25.8 | |
| Anti-inductionModel Size=0.13B2025.09 | 26.2 | |
| BalancedModel Size=0.13B2025.09 | 26.2 |