Tokenization efficiency on Financial
408Token CountCSV
Evaluation results
| Method | Links | |
|---|---|---|
| CSVSerialization format=CSV, Tokenizer=o200k_base2026.04 | 408 | |
| MarkdownSerialization format=Markdown, Tokenizer=o200k_base2026.04 | 505 | |
| Zen GridSerialization format=Zen Grid, Tokenizer=o200k_base2026.04 | 516 | |
| JSON CompactSerialization format=JSON Compact, Tokenizer=o200k_base2026.04 | 643 | |
| YAMLSerialization format=YAML, Tokenizer=o200k_base2026.04 | 840 | |
| JSON PrettySerialization format=JSON Pretty, Tokenizer=o200k_base2026.04 | 1,023 |