Math Mistake Remediation on Math Mistake Remediation Dataset
94Mistake Identification (Yes)GPT-5
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5Evaluator=GPT-5, Model Type=Proprietary2026.06 | 94 | 0.86 | 5.14 | 81.37 | 9.42 | 9.21 | 90.15 | 5.35 | 4.5 | 76.23 | 10.28 | 13.49 | |
| GPT-5Evaluator=LoMTL, Model Type=Proprietary2026.06 | 85.08 | 1.56 | 13.36 | 66.15 | 3.56 | 30.29 | 63.92 | 12.69 | 23.39 | 20.49 | 9.35 | 70.16 | |
| GPT-4.1-nanoTraining Strategy=DPO, Data Version=V4, Evaluator=LoMTL, Model Type=Fine-tuned2026.06 | 84.62 | 15.38 | 0 | 78.99 | 21.01 | 0 | 85.5 | 14.5 | 0 | 96.75 | 3.25 | 0 | |
| GPT-4.1-nanoTraining Strategy=SFT, Data Version=V4, Evaluator=LoMTL, Model Type=Fine-tuned2026.06 | 83.7 | 12.98 | 3.31 | 73.76 | 16.85 | 9.39 | 80.66 | 16.3 | 3.04 | 97.51 | 1.93 | 0.55 | |
| Qwen3-8BTraining Strategy=DPO, Data Version=V4*, Evaluator=LoMTL, Model Type=Fine-tuned2026.06 | 82.35 | 12.16 | 5.49 | 68.24 | 20.39 | 11.37 | 74.51 | 19.61 | 5.88 | 92.16 | 5.49 | 2.35 | |
| GPT-4.1-nanoEvaluator=GPT-5, Model Type=Proprietary2026.06 | 81.58 | 6.58 | 11.84 | 42.11 | 26.32 | 31.58 | 93.42 | 3.95 | 2.63 | 48.68 | 39.47 | 11.84 | |
| TutorRL-7BEvaluator=LoMTL2026.06 | 81.25 | 14.37 | 4.38 | 56.88 | 31.87 | 11.25 | 63.75 | 33.12 | 3.12 | 91.88 | 7.5 | 0.62 | |
| Qwen3-8BTraining Strategy=SFT, Data Version=V4, Evaluator=LoMTL, Model Type=Fine-tuned2026.06 | 68.95 | 22.11 | 8.95 | 51.05 | 26.32 | 22.63 | 61.05 | 27.37 | 11.58 | 81.58 | 12.63 | 5.79 | |
| Qwen3-8BTraining Strategy=DPO, Data Version=V4*, Evaluator=GPT-5, Model Type=Fine-tuned2026.06 | 57.14 | 24.18 | 18.68 | 45.05 | 26.74 | 28.21 | 73.63 | 17.95 | 8.42 | 61.54 | 28.57 | 9.89 | |
| GPT-4.1-nanoEvaluator=LoMTL, Model Type=Proprietary2026.06 | 53.95 | 1.32 | 44.74 | 44.74 | 1.32 | 53.95 | 42.11 | 7.89 | 50 | 9.21 | 5.26 | 85.53 | |
| GPT-4.1-nanoTraining Strategy=SFT, Data Version=V4, Evaluator=GPT-5, Model Type=Fine-tuned2026.06 | 47.24 | 26.24 | 26.52 | 38.12 | 28.45 | 33.43 | 73.2 | 19.06 | 7.73 | 80.11 | 17.13 | 2.76 | |
| Qwen3-8BTraining Strategy=SFT, Data Version=V4, Evaluator=GPT-5, Model Type=Fine-tuned2026.06 | 42.63 | 23.16 | 34.21 | 28.95 | 25.79 | 45.26 | 61.58 | 22.11 | 16.32 | 57.89 | 27.37 | 14.74 | |
| GPT-4.1-nanoTraining Strategy=DPO, Data Version=V4, Evaluator=GPT-5, Model Type=Fine-tuned2026.06 | 34.62 | 34.62 | 30.77 | 27.51 | 39.35 | 33.14 | 46.45 | 33.14 | 20.41 | 38.17 | 45.56 | 16.27 | |
| TutorRL-7BEvaluator=GPT-52026.06 | 27.5 | 35.62 | 36.88 | 10.62 | 30.63 | 58.75 | 66.88 | 29.38 | 3.75 | 87.5 | 10 | 2.5 | |
| SocraticLMEvaluator=LoMTL2026.06 | 5.42 | 88.33 | 6.25 | 5.42 | 62.92 | 31.67 | 15.42 | 80 | 4.58 | 92.08 | 7.08 | 0.83 | |
| SocraticLMEvaluator=GPT-52026.06 | 4.58 | 8.33 | 87.08 | 1.67 | 8.33 | 90 | 22.92 | 28.75 | 48.33 | 64.58 | 10 | 25.42 |