Tokenization Efficiency on 1.5-million sentence evaluation corpus (Sinhala)
17,360,196Other Token CountOpenAI (o200k)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OpenAI (o200k)2026.03 | 17,360,196 | — | 2.61 | |
| Llama 4 Scout2026.03 | 18,157,707 | — | 2.73 | |
| DeepSeek V32026.03 | 29,152,698 | — | 4.38 | |
| SGPE2026.03 | — | 6,654,288 | — |