Language Modeling on SlimPajama data-restricted regime 6B (test)
16.85Perplexity (Arxiv)TANDEM
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| TANDEMData Regime=Data-Restricted, Model Size=160M, Architecture=GPT-style, Metric Type=Perplexity2026.06 | 16.85 | 52.75 | 56.82 | 51.11 | 8.99 | 18.21 | 32.52 | 28.07 | |
| AioliData Regime=Data-Restricted, Model Size=160M, Architecture=GPT-style, Metric Type=Perplexity2026.06 | 17.68 | 62.48 | 69.02 | 61.44 | 9.26 | 17.79 | 33.06 | 30.67 | |
| DoGEData Regime=Data-Restricted, Model Size=160M, Architecture=GPT-style, Metric Type=Perplexity2026.06 | 17.76 | 60.88 | 65.08 | 58.81 | 9 | 17.71 | 33.94 | 30.1 | |
| UniformData Regime=Data-Restricted, Model Size=160M, Architecture=GPT-style, Metric Type=Perplexity2026.06 | 18.05 | 65.86 | 71.05 | 63.76 | 9.37 | 17.94 | 34.27 | 31.53 | |
| DoReMiData Regime=Data-Restricted, Model Size=160M, Architecture=GPT-style, Metric Type=Perplexity2026.06 | 18.9 | 80.29 | 89.05 | 79.02 | 10.24 | 19.74 | 43.2 | 36.91 | |
| Skill-ItData Regime=Data-Restricted, Model Size=160M, Architecture=GPT-style, Metric Type=Perplexity2026.06 | 20.93 | 52 | 57.11 | 49.5 | 8.77 | 16.74 | 40.49 | 29.24 |