Utterance Classification on Phase 2 debugging Overall 13 turns
91F1 ScoreClaude Sonnet
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Claude SonnetPrompt Condition=State-aug.2026.05 | 91 | 63 | |
| Claude SonnetPrompt Condition=Definitions2026.05 | 85 | 54 | |
| GPT-4oPrompt Condition=State-aug.2026.05 | 81 | 52 | |
| GPT-4oPrompt Condition=Definitions2026.05 | 74 | 40 | |
| GPT-4oPrompt Condition=Minimal2026.05 | 69 | 31 | |
| Claude SonnetPrompt Condition=Minimal2026.05 | 66 | 11 | |
| Llama-3.1-8B-InstructPrompt Condition=State-aug.2026.05 | 65 | 32 | |
| Qwen2.5-7B-InstructPrompt Condition=State-aug.2026.05 | 59 | 29 | |
| Qwen2.5-7B-InstructPrompt Condition=Definitions2026.05 | 57 | 26 | |
| Llama-3.1-8B-InstructPrompt Condition=Definitions2026.05 | 55 | 20 | |
| Qwen2.5-7B-InstructPrompt Condition=Minimal2026.05 | 54 | 29 | |
| Llama-3.1-8B-InstructPrompt Condition=Minimal2026.05 | 54 | 12 |