Tokenization Efficiency on 1.5-million sentence evaluation corpus Overall
68,739,586Other TokensDeepSeek V3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeek V32026.03 | 68,739,586 | — | 2.52 | |
| Llama 4 Scout2026.03 | 45,236,671 | — | 1.66 | |
| OpenAI (o200k)2026.03 | 43,174,798 | — | 1.58 | |
| SGPE2026.03 | — | 27,327,989 | — |