Language Modeling on wiki40b Hebrew (test)
1.27BPCUnigram
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| UnigramShortening=Dynamic2022.11 | 1.27 | 1.9 | |
| VanillaShortening=None2022.11 | 1.274 | 1 | |
| EntropyShortening=Dynamic2022.11 | 1.276 | 3.6 | |
| FixedShortening=Fixed, Fixed Factor=22022.11 | 1.279 | 2 | |
| GumbelShortening=Dynamic2022.11 | 1.281 | 4.7 | |
| WhitespacesShortening=Dynamic2022.11 | 1.284 | 3.8 | |
| FixedShortening=Fixed, Fixed Factor=32022.11 | 1.29 | 3 | |
| FixedShortening=Fixed, Fixed Factor=42022.11 | 1.304 | 4 |