Optical Reasoning on AquaRat, Gsm8k, GPQA Diamond, ScienceQA, and Zebra-Cot (test)
84.65Accuracy (AquaRat)Claude 4.5 (Text reasoning)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Claude 4.5 (Text reasoning)Setting=Text reasoning, Reasoning Tokens=257.8, Token Reduction=-0%2026.06 | 84.65 | 99.47 | 84.34 | 91.78 | 67 | 85.45 | 1.06 | |
| Claude 4.5 (T-OR)Setting=T-OR, Reasoning Tokens=207.4, Token Reduction=-20%2026.06 | 84.65 | 85.97 | 78.28 | 91.61 | 66.33 | 81.37 | 1.12 | |
| Claude 4.5 (T-OR)Setting=T-OR, Reasoning Tokens=155.6, Token Reduction=-40%2026.06 | 83.86 | 95.91 | 72.22 | 90.31 | 66 | 81.66 | 1.52 | |
| Claude 4.5 (T-OR)Setting=T-OR, Reasoning Tokens=259.3, Token Reduction=-0%2026.06 | 83.86 | 95.15 | 81.31 | 92.54 | 66 | 83.77 | 0.99 | |
| Claude 4.5 (T-OR)Setting=T-OR, Reasoning Tokens=103.7, Token Reduction=-60%2026.06 | 81.89 | 89.99 | 67.68 | 89.22 | 61 | 77.95 | 1.92 | |
| Claude 4.5 (T-OR)Setting=T-OR, Reasoning Tokens=51.8, Token Reduction=-80%2026.06 | 79.13 | 75.89 | 59.09 | 85.57 | 52 | 70.34 | 2.37 | |
| Gemini 2.5 Flash (T-OR)Setting=T-OR, Reasoning Tokens=155.6, Token Reduction=-40%2026.06 | 78.35 | 98.94 | 76.26 | 95.92 | 66 | 83.09 | 2 | |
| Claude 4.5 (No reasoning)Setting=No reasoning, Reasoning Tokens=0.0, Token Reduction=-100%2026.06 | 77.56 | 73.84 | 53.03 | 52.94 | 33 | 58.07 | — | |
| Kimi K2.5 (Text reasoning)Setting=Text reasoning, Reasoning Tokens=257.8, Token Reduction=-0%2026.06 | 76.77 | 99.62 | 75.25 | 93.14 | 70 | 82.96 | 1.07 | |
| Gemini 2.5 Flash (T-OR)Setting=T-OR, Reasoning Tokens=51.8, Token Reduction=-80%2026.06 | 76.77 | 98.94 | 78.28 | 95.97 | 64 | 82.79 | 5.95 | |
| Qwen3-VL-235B (T-OR)Setting=T-OR, Reasoning Tokens=155.6, Token Reduction=-40%2026.06 | 76.38 | 97.04 | 66.16 | 94.77 | 63 | 79.47 | 1.99 | |
| Kimi K2.5 (T-OR)Setting=T-OR, Reasoning Tokens=259.3, Token Reduction=-0%2026.06 | 75.98 | 99.32 | 74.24 | 95.04 | 71.33 | 83.18 | 1.07 | |
| Qwen3-VL-235B (T-OR)Setting=T-OR, Reasoning Tokens=207.4, Token Reduction=-20%2026.06 | 75.59 | 98.03 | 70.2 | 94.77 | 65.67 | 80.85 | 1.56 | |
| Kimi K2.5 (T-OR)Setting=T-OR, Reasoning Tokens=155.6, Token Reduction=-40%2026.06 | 75.2 | 98.94 | 70.71 | 93.41 | 69.33 | 81.52 | 1.68 | |
| Gemini 2.5 Flash (T-OR)Setting=T-OR, Reasoning Tokens=103.7, Token Reduction=-60%2026.06 | 75.2 | 99.09 | 73.23 | 95.92 | 63.67 | 81.42 | 2.84 | |
| Qwen3-VL-235B (T-OR)Setting=T-OR, Reasoning Tokens=103.7, Token Reduction=-60%2026.06 | 75.2 | 95.53 | 65.66 | 95.26 | 62 | 78.73 | 2.91 | |
| Kimi K2.5 (T-OR)Setting=T-OR, Reasoning Tokens=51.8, Token Reduction=-80%2026.06 | 74.41 | 83.4 | 56.57 | 87.2 | 67.33 | 73.78 | 3.55 | |
| Qwen3-VL-235B (Text reasoning)Setting=Text reasoning, Reasoning Tokens=257.8, Token Reduction=-0%2026.06 | 74.41 | 99.7 | 77.27 | 96.68 | 66 | 82.81 | 1.33 | |
| Qwen3-VL-235B (T-OR)Setting=T-OR, Reasoning Tokens=259.3, Token Reduction=-0%2026.06 | 74.41 | 97.95 | 73.23 | 94.99 | 65.67 | 81.25 | 1.26 | |
| Gemini 2.5 Flash (T-OR)Setting=T-OR, Reasoning Tokens=207.4, Token Reduction=-20%2026.06 | 74.02 | 93.1 | 70.71 | 96.13 | 66 | 79.99 | 1.35 | |
| Gemini 2.5 Flash (T-OR)Setting=T-OR, Reasoning Tokens=259.3, Token Reduction=-0%2026.06 | 73.62 | 89.23 | 78.28 | 96.13 | 64.67 | 80.39 | 1.1 | |
| Qwen3-VL-235B (T-OR)Setting=T-OR, Reasoning Tokens=51.8, Token Reduction=-80%2026.06 | 73.62 | 95.68 | 62.12 | 94.23 | 62.67 | 77.66 | 5.63 | |
| Gemini 2.5 Flash (Text reasoning)Setting=Text reasoning, Reasoning Tokens=257.8, Token Reduction=-0%2026.06 | 73.23 | 99.47 | 76.26 | 97.82 | 68.67 | 83.09 | 1.21 | |
| Kimi K2.5 (T-OR)Setting=T-OR, Reasoning Tokens=207.4, Token Reduction=-20%2026.06 | 72.83 | 99.24 | 76.26 | 94.55 | 69.33 | 82.45 | 1.3 | |
| Kimi K2.5 (T-OR)Setting=T-OR, Reasoning Tokens=103.7, Token Reduction=-60%2026.06 | 71.65 | 97.57 | 63.64 | 91.94 | 70.33 | 79.03 | 2.28 | |
| Kimi K2.5 (No reasoning)Setting=No reasoning, Reasoning Tokens=0.0, Token Reduction=-100%2026.06 | 69.69 | 61.79 | 46.46 | 64.11 | 35 | 55.41 | — | |
| Gemini 2.5 Flash (No reasoning)Setting=No reasoning, Reasoning Tokens=0.0, Token Reduction=-100%2026.06 | 68.9 | 59.67 | 42.93 | 67.37 | 26 | 51.95 | — | |
| GPT-5.1 (Text reasoning)Setting=Text reasoning, Reasoning Tokens=257.8, Token Reduction=-0%2026.06 | 68.11 | 89.08 | 74.75 | 96.46 | 57.33 | 77.15 | 1.27 | |
| GPT-5.1 (T-OR)Setting=T-OR, Reasoning Tokens=155.6, Token Reduction=-40%2026.06 | 60.44 | 93.34 | 76.26 | 96.9 | 61.33 | 77.65 | 2.14 | |
| Qwen3-VL-235B (No reasoning)Setting=No reasoning, Reasoning Tokens=0.0, Token Reduction=-100%2026.06 | 60.24 | 41.93 | 41.41 | 71.02 | 28 | 48.52 | — | |
| GPT-5.1 (T-OR)Setting=T-OR, Reasoning Tokens=207.4, Token Reduction=-20%2026.06 | 58.66 | 92.49 | 72.73 | 96.79 | 62.67 | 76.67 | 1.56 | |
| GPT-5.1 (T-OR)Setting=T-OR, Reasoning Tokens=259.3, Token Reduction=-0%2026.06 | 58.66 | 95 | 77.78 | 97.39 | 61.67 | 78.1 | 1.3 | |
| GPT-5.1 (T-OR)Setting=T-OR, Reasoning Tokens=103.7, Token Reduction=-60%2026.06 | 56.3 | 90.98 | 70.2 | 96.73 | 62.67 | 75.38 | 2.99 | |
| GPT-5.1 (T-OR)Setting=T-OR, Reasoning Tokens=51.8, Token Reduction=-80%2026.06 | 56.1 | 84.46 | 69.7 | 96.95 | 63 | 74.04 | 5.72 | |
| GPT-5.1 (No reasoning)Setting=No reasoning, Reasoning Tokens=0.0, Token Reduction=-100%2026.06 | 49.61 | 32.22 | 46.46 | 67.37 | 26.33 | 44.4 | — |