Compression on Goldfish data
-1.18English Compression Error RatePathPiece-BPE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| PathPiece-BPEtraining=monolingual FineWeb2026.06 | -1.18 | -2.14 | -2.87 | -2.71 | -2.59 | -2.69 | -2.37 | |
| ConvexToktraining=monolingual FineWeb2026.06 | -1.18 | -1.57 | -1.76 | -2.23 | -1.89 | -0.77 | -1.57 | |
| MinGram-PPtraining=monolingual FineWeb2026.06 | -1.16 | -2.1 | -2.85 | -2.66 | -2.56 | -2.67 | -2.33 | |
| MinGramtraining=monolingual FineWeb2026.06 | -0.86 | -1.49 | -1.68 | -2.37 | -2.18 | -2.4 | -1.83 | |
| Unigram-BPE-Inittraining=monolingual FineWeb2026.06 | -0.52 | -1.22 | -1.45 | -2.11 | -1.9 | -1.95 | -1.52 | |
| FSP-BPE-Inittraining=monolingual FineWeb2026.06 | -0.49 | -1.17 | -1.38 | -2.07 | -1.87 | -1.91 | -1.48 | |
| FSPtraining=monolingual FineWeb2026.06 | -0.33 | -0.72 | -1.16 | -1.05 | -1.1 | -1.4 | -0.96 | |
| BPEtraining=monolingual FineWeb2026.06 | 0.03 | 0.46 | 1.16 | -1.25 | -0.92 | -1.88 | -0.4 |