Language Modeling on PubMed OOD from The Pile
12.61PerplexityASEntmax
Evaluation results
| Method | Links | |
|---|---|---|
| ASEntmaxContext Length=8K2025.06 | 12.61 | |
| EntmaxContext Length=8K2025.06 | 12.86 | |
| ASEntmaxContext Length=16K2025.06 | 12.9 | |
| EntmaxContext Length=16K2025.06 | 13.02 | |
| SSMaxContext Length=8K2025.06 | 13.75 | |
| SSMaxContext Length=16K2025.06 | 14.72 | |
| ASEntmaxContext Length=4K2025.06 | 14.76 | |
| EntmaxContext Length=4K2025.06 | 14.79 | |
| SSMaxContext Length=4K2025.06 | 15.14 | |
| SoftmaxContext Length=8K2025.06 | 15.31 | |
| SoftmaxContext Length=4K2025.06 | 15.59 | |
| SoftmaxContext Length=16K2025.06 | 18.23 |