WikiText-2
Benchmarks
TaskDatasetSOTA resultTrendResults
WikiText-2 context length 4096 (test)
5.11PPL (WikiText-2)
15
Wikitext 2
4.67Perplexity (Wikitext-2)
12
WikiText-2 2017 (test)
7.5PPL (Uniform)
12
WikiText-2
5.44Perplexity (Baseline)
12
Wikitext-2 Standard (val)
40.2Perplexity
12
WikiText-2 (val)
5.486Min Validation Loss
11
Wikitext-2 raw v1
2.241Loss
10
WikiText-2
4.74Perplexity
9
WikiText-2 CTX=2048
6.1Perplexity
8
WikiText-2 stride (test)
3.504bpw
8
WikiText-2 raw-v1 (test)
2.5Delta PPL (n=1)
8
WikiText-2
5.03Original Perplexity (PPL)
8
WikiText-2 context length 2048 (test)
7.15Perplexity
7
Wikitext-2 word-level (dev)
66.9PPL
7
Wikitext-2
7.43Perplexity (Original)
6
WikiText-2 TinyLlama
0.004ΔPPL
6
WikiText-2 Mistral-7B
0.001ΔPPL
6
WikiText-2 BPE-2048 (val)
29.17Tail Accuracy
5
WikiText-2 Llama-3.1-8B-Instruct
3.8Delta PPL (%)
5
WikiText-2 (average)
25.07Average PPL
5
WikiText-2 50 seq × 256 tok (test)
14.94PPL
5
WikiText-2 Shifted
98.2Shifted PPL
5
WikiText-2 raw-v1 (val)
2.744Cross Entropy (CE)
5
WikiText-2 context length 8192 (test)
6.5Perplexity
5
WikiText-2 Top 5% most uncertain tokens (test)
5.12NLL
5