ResearchBenchmarksLanguage Modeling on SlimPajama 6B (Source Breakdown)Language Modeling on SlimPajama 6B (Source Breakdown)Follow10.26Arxiv PerplexityTANDEM10.259210.264610.2710.2754Jun 3, 2026Evaluation resultsMethodMethodLinksArxiv PerplexityBook PerplexityC4 PerplexityCC PerplexityGithub PerplexityStackexchange PerplexityWikipedia PerplexityAverage PerplexityTANDEMData scale=6B tokensData scale=6B tokens2026.0610.2629.5932.5229.526.0110.6816.0216.25UniformData scale=6B tokensData scale=6B tokens2026.0610.2832.3738.5334.575.8610.7615.2617.09