Language Modeling on The Pile Github (test)
40.88Bits Per Byte (BPB)HullFT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HullFTTime Budget (T)=2.0s, Average Selection Size (N*)=222026.05 | 40.88 | — | 4.73 | |
| HullFTTime Budget (T)=1.75s, Average Selection Size (N*)=192026.05 | 42.98 | — | 5.44 | |
| SIFTTime Budget (T)=2.0s, Average Selection Size (N*)=132026.05 | 45.61 | — | — | |
| kNNTime Budget (T)=2.0s, Average Selection Size (N*)=172026.05 | 46.97 | — | — | |
| kNNTime Budget (T)=1.75s, Average Selection Size (N*)=152026.05 | 48.42 | — | — | |
| SIFTTime Budget (T)=1.75s, Average Selection Size (N*)=112026.05 | 48.95 | — | — | |
| Split ModelModel structure=Split, Total compute budget=720B tokens, Number of parameters=1.3B, ts=340B2026.03 | — | 3.42 | — | |
| Standalone Pretrained ModelModel structure=Standalone, Total compute budget=720B tokens, Number of parameters=1.3B2026.03 | — | 3.74 | — |