ResearchTasksPretokenization Offsets CalculationPretokenization Offsets CalculationFollowBenchmarksDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyDatasetSOTA methodMetricTrendResultsLast updatedLLaMA-3 Tokenizer Evaluation DataPeek246.037Throughput (Mbps)4Feb 26, 2026