SlimPajama
Benchmarks
TaskDatasetSOTA resultTrendResults
SlimPajama
2.97Perplexity (PPL)
77
SlimPajama latest (val)
3.078Validation Loss
26
SlimPajama (test)
0.54PPL (CommonCrawl)
25
SlimPajama data-restricted
24.35Average Perplexity
17
SlimPajama (val)
3.09Perplexity
13
SlimPajama large-scale (train)
2.129L(ψ)
8
SlimPajama Data Abundant Regime 6B (test)
11.46Arxiv Perplexity
6
SlimPajama data-restricted regime 6B (test)
16.85Perplexity (Arxiv)
6
SlimPajama-672B (val)
8.09Validation Perplexity
6
SlimPajama 6B (val)
2.3Validation Loss
4
SlimPajama-6B (train)
2.1Train Loss
4
SlimPajama
63.5Pointwise Score
3
SlimPajama Quality Evaluation (test)
86.3Gen Quality (Std. Prefix)
3
SlimPajama 10M (dev)
9.219Perplexity
3
SlimPajama 6B tokens
10.26Arxiv Perplexity
2
SlimPajama
6Tie Percentage
2