Document and chart understanding on CharXiv DQ
98.4Pass@1Kimi K2.5 + ETCHR
Evaluation Results
| Method | Links | |
|---|---|---|
| Kimi K2.5 + ETCHRTemperature=02026.05 | 98.4 | |
| Kimi K2.5Temperature=02026.05 | 98.3 | |
| Gemini-3.1-Flash-Lite + ETCHRTemperature=02026.05 | 94.6 | |
| Gemini 1.5 Prothinking=true, decoding=greedy2025.05 | 94.4 | |
| Gemini-3-Pro High2026.06 | 94.4 | |
| Gemini-3.1-Flash-LiteTemperature=02026.05 | 93.8 | |
| GPT-5.2 High2026.06 | 93.8 | |
| Seed2.0 Pro2026.06 | 93.5 | |
| Seed2.0 Lite2026.06 | 93.3 | |
| Claude-Opus-4.52026.06 | 92.7 | |
| Seed 1.5-VLthinking=true, decoding=greedy2025.05 | 92.6 | |
| Seed 1.5-VLthinking=false, decoding=greedy2025.05 | 92.6 | |
| Claude 3.7 Sonnetthinking=true, decoding=sampling2025.05 | 92 | |
| Seed2.0 Mini2026.06 | 91.9 | |
| OpenAI o1thinking=true, decoding=greedy2025.05 | 88.9 | |
| Seed1.82026.06 | 88 | |
| Qwen 2.5-VL 72Bthinking=false, decoding=greedy2025.05 | 87.4 | |
| Qwen3-VL-8B + ETCHRTemperature=02026.05 | 86.8 | |
| GPT-4othinking=false, decoding=greedy2025.05 | 86.5 | |
| Qwen3-VL-8BTemperature=02026.05 | 85 | |
| DeepEyesV2Temperature=0, Max tool invocations=102026.05 | 78.6 | |
| ThymeTemperature=0, Max tool invocations=102026.05 | 66.1 | |
| Bagel-Zebra-CoTTemperature=0, Max image generations=102026.05 | 62.8 | |
| ThinkMorph-7BTemperature=0, Max image generations=102026.05 | 61.2 |