Reasoning on ProsQA
100AccImgCoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ImgCoTBackbone=Qwen2.5-1.5B-Instruction2026.01 | 100 | 8 | |
| ImgCoT (w/o layout)Backbone=Qwen2.5-1.5B-Instruction2026.01 | 100 | 8 | |
| L-ImgCoTBackbone=Qwen2.5-1.5B-Instruction2026.01 | 100 | 21.7 | |
| Full-CoTBackbone=LLama3.2-3B-Instruction2026.01 | 100 | 41.3 | |
| CoconutBackbone=LLama3.2-3B-Instruction2026.01 | 100 | 6 | |
| CoLaRBackbone=LLama3.2-3B-Instruction2026.01 | 100 | 6.8 | |
| ImgCoTBackbone=LLama3.2-3B-Instruction2026.01 | 100 | 8 | |
| ImgCoT (w/o layout)Backbone=LLama3.2-3B-Instruction2026.01 | 100 | 8 | |
| ImgCoT (w/ textual tokens)Backbone=LLama3.2-3B-Instruction2026.01 | 100 | 8 | |
| L-ImgCoTBackbone=LLama3.2-3B-Instruction2026.01 | 100 | 27.5 | |
| Full-CoTBackbone=Qwen2.5-1.5B-Instruction2026.01 | 99.6 | 46.2 | |
| CoconutBackbone=Qwen2.5-1.5B-Instruction2026.01 | 99.6 | 6 | |
| CODIBackbone=Qwen2.5-1.5B-Instruction2026.01 | 99.6 | 6 | |
| CoLaRBackbone=Qwen2.5-1.5B-Instruction2026.01 | 99.4 | 7.1 | |
| ImgCoT (w/ textual tokens)Backbone=Qwen2.5-1.5B-Instruction2026.01 | 99.2 | 8 | |
| ICoTBackbone=LLama3.2-3B-Instruction2026.01 | 99.2 | 0 | |
| L-ImgCoTBackbone=Qwen2.5-0.5B-Instruction2026.01 | 98.6 | 40.9 | |
| ICoTBackbone=Qwen2.5-1.5B-Instruction2026.01 | 98.6 | 0 | |
| iCoTBackbone=GPT-2, Evaluation Stage=stage 62026.06 | 98.2 | — | |
| ImgCoTBackbone=Qwen2.5-0.5B-Instruction2026.01 | 97.4 | 8 | |
| ImgCoT (w/o layout)Backbone=Qwen2.5-0.5B-Instruction2026.01 | 96.8 | 8 | |
| ImgCoT (w/ textual tokens)Backbone=Qwen2.5-0.5B-Instruction2026.01 | 96.2 | 8 | |
| AGCLRBackbone=GPT-2, Evaluation Stage=stage 62026.06 | 96 | — | |
| Full-CoTBackbone=Qwen2.5-0.5B-Instruction2026.01 | 94.6 | 71.3 | |
| Vanilla CoCoNuTBackbone=GPT-2, Evaluation Stage=stage 62026.06 | 92 | — | |
| No-CoTBackbone=GPT-2, Evaluation Stage=stage 62026.06 | 76.7 | — | |
| Pause TokenBackbone=GPT-2, Evaluation Stage=stage 62026.06 | 75.9 | — | |
| CoconutBackbone=Qwen2.5-0.5B-Instruction2026.01 | 73.8 | 6 | |
| CoLaRBackbone=Qwen2.5-0.5B-Instruction2026.01 | 68.9 | 18.9 | |
| CoTBackbone=GPT-2, Evaluation Stage=stage 62026.06 | 55 | — | |
| ICoTBackbone=Qwen2.5-0.5B-Instruction2026.01 | 54.9 | 0 | |
| CODIBackbone=Qwen2.5-0.5B-Instruction2026.01 | 0 | 6 |