Language Modeling on WikiText-2 byte-level (val)
3.595BPBGated Hybrid Retrieval
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gated Hybrid Retrievald=64, setup=80-step LM2026.04 | 3.595 | 5,659 | 4,910 | |
| Triple-Hybridd=64, setup=80-step LM2026.04 | 4.843 | 18,433 | 18,870 | |
| Transformerd=64, setup=80-step LM2026.04 | 5.144 | 602,684 | 344,185 |