Tokenization Efficiency on Indic-English (evaluation)
100NSL ScoreSarvam
Evaluation results
| Method | Links | |
|---|---|---|
| Sarvam2025.11 | 100 | |
| Sarvam2025.11 | 99 | |
| GPT-oss2025.11 | 96 | |
| GPT-oss2025.11 | 95 | |
| Sutra2025.11 | 93 | |
| Sutra2025.11 | 92 | |
| Sarvam2025.11 | 91 | |
| Gemma-32025.11 | 91 | |
| Gemma-32025.11 | 87 | |
| MUTANT-Indic2025.11 | 78 | |
| MUTANT-Indic2025.11 | 65 | |
| Gemma-32025.11 | 63 | |
| GPT-oss2025.11 | 63 | |
| GPT-oss2025.11 | 60 | |
| Sutra2025.11 | 55 | |
| MUTANT-Indic2025.11 | 45 | |
| Gemma-32025.11 | 44 | |
| Sutra2025.11 | 24 | |
| Sarvam2025.11 | 24 | |
| MUTANT-Indic2025.11 | 18 | |
| Sarvam2025.11 | 1.45 | |
| GPT-oss2025.11 | 1.43 | |
| Gemma-32025.11 | 0.59 | |
| MUTANT-Indic2025.11 | 0.45 | |
| Sutra2025.11 | 0.26 |