List your company

Reasoning Trajectory Generation on Socratic Debugging of Student Code 227 samples

1,962Reasoning Steps CountClaude Haiku-4.5
1,243.361,429.931,616.51,803.07Nov 1, 2025

Evaluation results

MethodLinks
Claude Haiku-4.5Reasoning=false2025.111,96262.6
GPT-5-miniEffort level=minimal-effort, Reasoning=true2025.111,82668.3
Gemini 2.5-flashReasoning=true2025.111,82682.7
Claude Sonnet-4.5Reasoning=false2025.111,79280.6
Claude Sonnet-4.5Reasoning=true2025.111,77687.2
Claude Haiku-4.5Reasoning=true2025.111,73878.9
Gemini 2.5-proReasoning=true2025.111,62885.3
GPT-5Effort level=minimal-effort, Reasoning=true2025.111,57785
GPT-5Effort level=low-effort, Reasoning=true2025.111,48890.7
GPT-5-miniEffort level=low-effort, Reasoning=true2025.111,45359.5
Gemini 2.5-proReasoning=false2025.111,43977.2
Gemini 2.5-flashReasoning=false2025.111,37983.5
GPT-5-miniEffort level=medium-effort, Reasoning=true2025.111,35168.9
GPT-5Effort level=medium-effort, Reasoning=true2025.111,27191.1