Downstream Repair on Arithmetic trace-level 60/20/20 (test)
92.4Repair AccuracyCROP
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CROPRepair model=Qwen2.5-7B-Instruct, Alpha=0.05, Input Strategy=CROP prefix2026.05 | 92.4 | 1.02 | 0.28 | |
| Full traceRepair model=Qwen2.5-7B-Instruct, Alpha=0.05, Input Strategy=Full trace2026.05 | 90.8 | — | — | |
| Whole-trace abst.Repair model=Llama3.1-8B, Alpha=0.05, Input Strategy=Whole-trace abst.2026.05 | 90.5 | — | — | |
| Whole-trace abst.Repair model=Qwen2.5-7B-Instruct, Alpha=0.05, Input Strategy=Whole-trace abst.2026.05 | 90.3 | — | — | |
| CROPRepair model=Llama3.1-8B, Alpha=0.05, Input Strategy=CROP prefix2026.05 | 89.4 | -1.18 | -1.84 | |
| CROPRepair model=Gemma 4 E4B IT, Alpha=0.05, Input Strategy=CROP prefix2026.05 | 88.7 | 5.32 | 4.41 | |
| Full traceRepair model=Llama3.1-8B, Alpha=0.05, Input Strategy=Full trace2026.05 | 87.3 | — | — | |
| Question onlyRepair model=Qwen2.5-7B-Instruct, Alpha=0.05, Input Strategy=Question only2026.05 | 85.5 | — | — | |
| CROPRepair model=DeepSeek-R1-0528-Qwen3-8B, Alpha=0.05, Input Strategy=CROP prefix2026.05 | 85 | 5.53 | 4.79 | |
| Whole-trace abst.Repair model=Gemma 4 E4B IT, Alpha=0.05, Input Strategy=Whole-trace abst.2026.05 | 83.4 | — | — | |
| Question onlyRepair model=Gemma 4 E4B IT, Alpha=0.05, Input Strategy=Question only2026.05 | 80.5 | — | — | |
| Full traceRepair model=Gemma 4 E4B IT, Alpha=0.05, Input Strategy=Full trace2026.05 | 79.9 | — | — | |
| Whole-trace abst.Repair model=DeepSeek-R1-0528-Qwen3-8B, Alpha=0.05, Input Strategy=Whole-trace abst.2026.05 | 78.8 | — | — | |
| Full traceRepair model=DeepSeek-R1-0528-Qwen3-8B, Alpha=0.05, Input Strategy=Full trace2026.05 | 78.5 | — | — | |
| Question onlyRepair model=Llama3.1-8B, Alpha=0.05, Input Strategy=Question only2026.05 | 73.7 | — | — | |
| Question onlyRepair model=DeepSeek-R1-0528-Qwen3-8B, Alpha=0.05, Input Strategy=Question only2026.05 | 63.9 | — | — |