Language Modeling on FineWeb-Edu T=2048
41.65Perplexityfib / stagger W=12
Evaluation Results
| Method | Links | |
|---|---|---|
| fib / stagger W=12Training Length=10242026.06 | 41.65 | |
| fib / stagger W=8Training Length=10242026.06 | 42.17 | |
| fib / coprime W=12Training Length=10242026.06 | 42.53 | |
| fib / learned W=8Training Length=10242026.06 | 43.34 | |
| fib / fixed W=8Training Length=10242026.06 | 43.74 | |
| pow2 / fixed W=8Training Length=10242026.06 | 44.98 | |
| denseTraining Length=10242026.06 | 49.15 |