ResearchBenchmarksSentence Retrieval on 20 languages Different scriptFollow44.1AccuracyUnigram42.74843.09943.4543.801May 26, 2023Evaluation ResultsMethodMethodLinksAccuracyUnigramTokenizer=UnigramTokenizer=Unigram2023.0544.1BPETokenizer=BPETokenizer=BPE2023.0544.1TokMixTokenizer=TokMixTokenizer=TokMix2023.0542.8