Tokenization Efficiency on 1.5-million sentence evaluation corpus English
7,904,670Other Tokens CountDeepSeek V3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeek V32026.03 | 7,904,670 | — | 1.09 | |
| Llama 4 Scout2026.03 | 7,512,843 | — | 1.04 | |
| OpenAI (o200k)2026.03 | 7,420,527 | — | 1.02 | |
| SGPE2026.03 | — | 7,240,147 | — |