Language Modeling on Isabelle (test)
4.44PerplexityFOT
Evaluation results
| Method | Links | |
|---|---|---|
| FOTContext Length=64K, Zero-shot=true2023.07 | 4.44 | |
| FOTContext Length=16K, Zero-shot=true2023.07 | 4.51 | |
| FOTContext Length=4K, Zero-shot=true2023.07 | 4.93 | |
| FOTContext Length=2K, Zero-shot=true2023.07 | 5.63 | |
| Transformer-XLContext Length=2K, Zero-shot=true2023.07 | 5.76 | |
| Memorizing TransformerContext Length=64K, Zero-shot=true2023.07 | 6.64 | |
| Memorizing TransformerContext Length=16K, Zero-shot=true2023.07 | 6.66 | |
| Memorizing TransformerContext Length=4K, Zero-shot=true2023.07 | 6.93 | |
| Memorizing TransformerContext Length=2K, Zero-shot=true2023.07 | 7.34 |