OWT
Benchmarks
TaskDatasetSOTA resultTrendResults
OWT (test)
17.5Perplexity (PPL)
79
OWT
15.06Gen. PPL
78
OWT
5.33GPT2 Perplexity
41
OWT
19.99Perplexity (PPL)
24
OWT (val)
17.5PPL
23
OWT L=1024 (test)
20.96NELBO PPL
11
OWT L=1024
14.1Generative Perplexity
6
OWT
11.2PPL
6
OWT L=2048
13.2Generative Perplexity
5
OWT L=8192
54Generation Latency (s)
5
OWT L=2048
13.3Sampling Latency (s)
5
OWT Qwen-2 tokenizer (val)
21.9Validation Perplexity
4
OWT (test)
92.94MAUVE
3
OWT L=10240 (test)
23.4Gen. PPL
2