Socratic Conversation Generation on Socratic Debugging of Student Code 227 samples
98.7Valid Conversions RateGPT-5
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GPT-5Effort level=low-effort, Reasoning=true2025.11 | 98.7 | 99.4 | |
| GPT-5-miniEffort level=medium-effort, Reasoning=true2025.11 | 95.9 | 98.7 | |
| GPT-5Effort level=medium-effort, Reasoning=true2025.11 | 94.8 | 98.5 | |
| GPT-5Effort level=minimal-effort, Reasoning=true2025.11 | 94.3 | 98.6 | |
| Claude Sonnet-4.5Reasoning=true2025.11 | 92.5 | 97.9 | |
| GPT-5-miniEffort level=low-effort, Reasoning=true2025.11 | 92.1 | 98 | |
| Claude Sonnet-4.5Reasoning=false2025.11 | 89 | 97.4 | |
| Gemini 2.5-flashReasoning=false2025.11 | 86.1 | 97.4 | |
| Gemini 2.5-flashReasoning=true2025.11 | 85.8 | 96.7 | |
| GPT-5-miniEffort level=minimal-effort, Reasoning=true2025.11 | 85 | 96.9 | |
| Claude Haiku-4.5Reasoning=true2025.11 | 81.5 | 95.7 | |
| Gemini 2.5-proReasoning=true2025.11 | 78.7 | 95.5 | |
| Gemini 2.5-proReasoning=false2025.11 | 78.3 | 94.9 | |
| Claude Haiku-4.5Reasoning=false2025.11 | 68.3 | 93.2 |