OpenWebText
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
OpenWebText
1.07Gen. PPL
227
OpenWebText
5.33Perplexity
190
OpenWebText
3.18Perplexity
187
OpenWebText (val)
2.609Validation Loss
118
OpenWebText2 (test)
16.2Perplexity
104
OpenWebText
15.24Throughput (samples/GPU/s)
60
OpenWebText
11.312Gen PPL
54
OpenWebText unconditional generation L = 1024
100MAUVE
50
OpenWebText (OWT) (val)
7.77Perplexity
47
OpenWebText (OWT) L=1024 (held-out)
1MAUVE
45
OpenWebText
24.11Generation Perplexity (Gen.PPL)
42
OpenWebText L=1024 (test)
14.1Generation Perplexity
40
OpenWebText (test)
2.947Average Perplexity
40
OpenWebText
22.099Generative Perplexity
30
OpenWebText (OWT) (test)
27.35Generation Perplexity
30
OpenWebText
18.68PPL
30
OpenWebText (held-out)
18.53ID PPL
30
OpenWebText
0.622Clustering Score
30
OpenWebText Neutral to Negative (test)
12.48Perplexity (PPL)
27
OpenWebText Neutral to Positive (test)
12.48Perplexity (PPL)
27
OpenWebText2
5.45Perplexity (PPL)
25
OpenWebText (train)
2.524Train Loss
25
NanoGPT OpenWebText
391,100Throughput (tokens/s)
24
OpenWebText (OWT) (test)
17.54Perplexity
22
OpenWebText (test)
692.3LLAMA2 Score
21