Language Modeling on The Pile ArXiv (test)
88.57BPB ScoreHullFT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HullFTTime Budget (T)=2.0s, Average Selection Size (N*)=222026.05 | 88.57 | — | 2.25 | |
| HullFTTime Budget (T)=1.75s, Average Selection Size (N*)=192026.05 | 89.39 | — | 1.94 | |
| kNNTime Budget (T)=2.0s, Average Selection Size (N*)=172026.05 | 90.82 | — | — | |
| SIFTTime Budget (T)=2.0s, Average Selection Size (N*)=132026.05 | 90.84 | — | — | |
| kNNTime Budget (T)=1.75s, Average Selection Size (N*)=152026.05 | 91.33 | — | — | |
| SIFTTime Budget (T)=1.75s, Average Selection Size (N*)=112026.05 | 91.8 | — | — | |
| Split ModelModel structure=Split, Total compute budget=720B tokens, Number of parameters=1.3B, ts=340B2026.03 | — | 9.92 | — | |
| Standalone Pretrained ModelModel structure=Standalone, Total compute budget=720B tokens, Number of parameters=1.3B2026.03 | — | 11.01 | — |