Zero-shot Evaluation on DCLM CORE
0.268CORE ScoreMinGram-PP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MinGram-PPTokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.268 | — | |
| FSP-BPE-InitTokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.2664 | -0.61 | |
| MinGramTokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.2658 | -0.82 | |
| Unigram-BPE-InitTokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.2658 | -0.81 | |
| FSPTokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.2651 | -1.1 | |
| PathPiece-BPETokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.2643 | -1.38 | |
| ConvexTokTokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.2631 | -1.83 | |
| UnigramTokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.2625 | -2.05 | |
| BPETokenizer training data=FineWeb-English, Model architecture=depth-24 nanochat, Training tokens=5.86B, Number of seeds=202026.06 | 0.2605 | -2.78 |