Language Modeling on SlimPajama Data Abundant Regime 6B (test)
11.46Arxiv PerplexityUniform
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| UniformModel Scale=160M, Training Regime=Data Abundant Regime, Mixture Ratio Source=Uniform2026.06 | 11.46 | 62.53 | 66.43 | 59.29 | 6.71 | 13.98 | 28.31 | 25.74 | |
| AioliModel Scale=160M, Training Regime=Data Abundant Regime, Mixture Ratio Source=Aioli [2]2026.06 | 11.47 | 61.89 | 65.52 | 58.24 | 6.74 | 14.08 | 28.48 | 25.66 | |
| TANDEMModel Scale=160M, Training Regime=Data Abundant Regime2026.06 | 11.53 | 61.82 | 65.92 | 58.86 | 6.63 | 13.76 | 27.26 | 25.43 | |
| Skill-ItModel Scale=160M, Training Regime=Data Abundant Regime, Mixture Ratio Source=Aioli [2]2026.06 | 11.76 | 62.24 | 64.58 | 59.84 | 6.36 | 12.36 | 34.87 | 25.87 | |
| DoReMiModel Scale=160M, Training Regime=Data Abundant Regime, Mixture Ratio Source=Aioli [2]2026.06 | 12.71 | 80.09 | 82.6 | 71.76 | 5.75 | 14.26 | 29.54 | 28.32 | |
| DoGEModel Scale=160M, Training Regime=Data Abundant Regime, Mixture Ratio Source=Aioli [2]2026.06 | 12.89 | 51.5 | 54.32 | 49.34 | 8.48 | 16.77 | 37.21 | 26.6 |