Tokenizer Compression on Reasoning
3.51Bits per TokenQwen 3
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen 3Vocabulary Size=152k2026.02 | 3.51 | |
| Llama 3Vocabulary Size=128k2026.02 | 3.59 | |
| DeepSeek R1Vocabulary Size=128k2026.02 | 3.61 | |
| GPT-OSSVocabulary Size=200k2026.02 | 3.61 | |
| TrinityVocabulary Size=200k2026.02 | 3.67 |