List your company

ResearchFormerly WizWand

Tokenization Intrinsic Evaluation

Benchmarks

DatasetSOTA methodMetricTrendResultsLast updated

FLORES+ 30-lang

PA-BPE (ratios)

0.027Compression Rate

6

Jul 7, 2026

mC4 unbalanced 60-lang (test)

0.021Compression Rate

5

Jul 7, 2026