Language Modeling on FineWeb-Edu T=1024
33.37Perplexitydense
Evaluation Results
| Method | Links | |
|---|---|---|
| denseTraining Length=10242026.06 | 33.37 | |
| fib / stagger W=12Training Length=10242026.06 | 42.02 | |
| fib / coprime W=12Training Length=10242026.06 | 42.09 | |
| fib / stagger W=8Training Length=10242026.06 | 42.53 | |
| fib / learned W=8Training Length=10242026.06 | 43.7 | |
| fib / fixed W=8Training Length=10242026.06 | 44.08 | |
| pow2 / fixed W=8Training Length=10242026.06 | 44.78 |