Penn Treebank
Benchmarks
TaskDatasetSOTA resultTrendResults
Penn Treebank (test)
46.34Perplexity
450
Penn Treebank (val)
46.64Perplexity
178
Penn Treebank (PTB) (test)
14.72Perplexity
130
Penn Treebank (test)
1.175BPC
113
Penn Treebank (PTB) (test)
96.4LAS
80
Penn Treebank word-level (test)
49.95Perplexity
72
Penn Treebank (PTB) (val)
46.63Perplexity
70
Penn Treebank
11.18Perplexity
69
Penn TreeBank (test)
97.96Accuracy
64
Penn Treebank WSJ (section 23 test)
95.8F1 Score
55
Penn Treebank (PTB)
30.44Throughput (Bytes/sec)
27
Penn Treebank char-level (test)
1.16BPC
25
Penn Treebank WSJ (test)
84.3F1 (mean)
25
Penn Treebank (PTB) Section 23 v2.2 (test)
95.66UAS
17
Penn TreeBank English (test)
69.6Mean S-F1
16
Penn Treebank WSJ Section 23 (test)
57.22F1 Score
15
Penn Treebank (PTB) Section 23 v2.2 (test)
97.97POS Accuracy
15
Penn Treebank shortest 25% of samples < 128 tokens (test)
76.6Bracket Precision
14
Penn Treebank (dev)
56.5Perplexity (PPL)
14
Penn Treebank WSJ10 (test)
82.9F1 (max)
14
English Penn Treebank (test)
1,127Speed (Sents/s)
11
Penn Treebank (PTB) word-level (val)
56.5Perplexity
11
Penn Treebank (test)
34.5BLEU
11
Penn Treebank POS (test)
97.58F1 Score
10
Penn Treebank character-level (val)
1.24BPC
10