LM1B
Benchmarks
TaskDatasetSOTA resultTrendResults
LM1B (test)
20.86Perplexity
206
LM1B
48.5PPL (Generalized)
93
LM1B (test)
2.46Entropy
90
LM1B (val)
16.57Perplexity
67
LM1B
22.8Perplexity
65
LM1B sequence length 128
40.2Generation Perplexity (PPL)
43
LM1B
68.11Perplexity (PPL)
39
LM1B
36.42Generation Perplexity
31
LM1B zero-shot
51.25Perplexity
30
LM1B
4.29Entropy
24
PD1-LM1B (val)
0.628Validation Error
24
LM1B 1024 sequences of length 128
186.79Generative PPL
20
LM1B L=128 (test)
24.53NELBO PPL
17
LM1B (test)
8.94Block Efficiency
15
LM1B (test)
7.88BE
10
LM1B GPT-2 small model size equivalent (test)
20.53Perplexity
10
LM1B
21.5PPL
7
LM1B GPT2
65.629PPL
4
LM1B ctx len. 128 (val)
25.72PPL (Val)
3
LM1B (val)
51.25Perplexity
1
LM1B 1.0 (test)
—Primary metric
0