Reasoning on CogniLoad Outcome suite 1.0 (m=10 m=100)
1.26E0 ScoreDS-R1D-Qwen-1.5B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DS-R1D-Qwen-1.5Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 1.26 | 19 | 14,690 | 94 | 20 | |
| MiMo-7B-RLprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 2.99 | 6 | 2,048 | 100 | 6 | |
| Phi-4-reasoning-plusprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 3.08 | 46 | 15,046 | 79 | 58 | |
| EXAONE-Deep-32Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 3.14 | 35 | 11,060 | 100 | 35 | |
| Phi-4-mini-reasoningprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 3.21 | 26 | 8,005 | 100 | 26 | |
| DS-R1D-Qwen-7Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 3.84 | 26 | 6,858 | 100 | 26 | |
| Phi-4-reasoningprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 4.24 | 51 | 12,013 | 88 | 58 | |
| gemini-2.5-flashprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 4.62 | 66 | 14,332 | 72 | 92 | |
| gemini-2.5-proprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 5.21 | 75 | 14,416 | 77 | 97 | |
| Qwen3-1.7Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 5.3 | 21 | 3,949 | 100 | 21 | |
| GLM-4.5-airprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 5.91 | 59 | 9,935 | 94 | 63 | |
| gpt-5-nano(08-07)prompting=zero-shot, token budget=32,768, decoding=default2026.02 | 5.93 | 46 | 7,792 | 100 | 46 | |
| QwQ-32Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 6.18 | 50 | 8,168 | 97 | 52 | |
| Qwen3-8Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 6.45 | 41 | 6,405 | 96 | 43 | |
| DS-R1-0528prompting=zero-shot, token budget=32,768, decoding=default2026.02 | 7.18 | 68 | 9,476 | 100 | 68 | |
| DS-R1D-Qwen-32Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 8.03 | 51 | 6,393 | 100 | 51 | |
| Qwen3-30B-A3Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 8.77 | 45 | 5,105 | 99 | 45 | |
| gpt-5-mini-(08-07)prompting=zero-shot, token budget=32,768, decoding=default2026.02 | 8.91 | 79 | 8,878 | 97 | 82 | |
| gemini-2.5-flash-liteprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 11.36 | 38 | 3,339 | 99 | 38 | |
| Qwen3-32Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 11.47 | 50 | 4,383 | 100 | 50 | |
| GLM-Z1-32B-0414prompting=zero-shot, token budget=32,768, decoding=default2026.02 | 11.94 | 48 | 3,980 | 100 | 48 | |
| o4-mini(04-16)prompting=zero-shot, token budget=32,768, decoding=default2026.02 | 12.27 | 76 | 6,232 | 98 | 78 | |
| DS-R1D-Llama-70Bprompting=zero-shot, token budget=32,768, decoding=default2026.02 | 12.57 | 58 | 4,592 | 100 | 58 | |
| gpt-5(08-07)prompting=zero-shot, token budget=32,768, decoding=default2026.02 | 13.41 | 92 | 6,897 | 98 | 95 | |
| o3(04-16)prompting=zero-shot, token budget=32,768, decoding=default2026.02 | 15.25 | 89 | 5,869 | 98 | 91 |