LAMBADA
Benchmarks
TaskDatasetSOTA resultTrendResults
Lambada clean OpenAI (test)
0.436Bits Per Token (bpt)
6
LAMBADA
0.582Cross-Entropy (CE)
6
Lambada S
18.43Accuracy
6
Lambada
43.7Compressed Accuracy
6
LAMBADA Epoch 2
27.9LAM Accuracy
5
LAMBADA
24.16Accuracy
5
LAMBADA
7.02Perplexity
5
LAMBADA 500 examples
23Accuracy (LAMBADA 500)
4
Lambada (val)
15.56Perplexity
4
LAMBADA (dev)
58.31Accuracy
4
LAMBADA (control)
48.01Accuracy
4
LAMBADA
0.021CF Score
3
LAMBADA
44.36LAMBADA Accuracy
3
LAMBADA 5-shot
63.4Accuracy (LAMBADA 5-shot)
3
LAMBADA full (test val)
80.6Accuracy
3
LAMBADA first 1000 examples (standard)
20.3Accuracy (LAMBADA)
3
Lambada German 5-shot (test)
47.1Accuracy
3
LAMBADA medium model (test)
61.3Accuracy
3
LAMBADA context (test)
68.88Accuracy
3
LAMBADA
10.98Perplexity (PPL)
2
LAMBADA
0.004MAE
2
LAMBADA (val)
51.28Perplexity (PPL)
1