ResearchDatasetsLLaMA TokenizerFollowBenchmarksTask NameDataset NameSOTA MethodSortMost resultsRecently updatedMost papersApplyTask NameDataset NameSOTA ResultTrendResultsPretokenization Offsets CalculationLLaMA-3 Tokenizer Evaluation Data46.037Throughput (Mbps)4