WikiText
Benchmarks
TaskDatasetSOTA resultTrendResults
WikiText-2 (test)
1.2PPL
2,416
WikiText
0.284PPL
740
WikiText
2.9Word Perplexity
331
Wikitext
7.21Wikitext PPL
151
WikiText-2
4.88Perplexity
105
WikiText (test)
4.88Perplexity
69
WikiText (val)
12.51Perplexity
62
WikiText
0.1Perplexity
47
WikiText v1 (test)
13.33Perplexity
30
WikiText (held-out)
9.8Perplexity (PPL)
25
Wikitext zero-shot
24.01Perplexity
22
WikiText-103
159,000Throughput (tokens/s)
21
WikiText
0.18Loss Barrier
20
WikiText
54.83Rep-2
18
WikiText-103 Clean (val)
31.82Perplexity
18
WikiText (WT)
31Relative PPL Change (%)
16
WikiText
1.7PPL Change (%)
16
WikiText-103
2Bits Per Character (BPC)
13
WikiText-2 vLLM harness (test)
8.87Perplexity (PPL)
12
WikiText
0Epsilon
12
Wikitext (test)
95Diversity (DIV)
12
WikiText
17.69PPL
11
WikiText-103 20w x 2048
9.603Perplexity (PPL)
10
WikiText (test)
38Time (s)
10
WikiText 1,000-example evaluation slice (test)
12.723Perplexity
9