Code Debugging on DebugBench
66.6Average AccuracyMOCHA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MOCHAEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 66.6 | — | — | — | |
| Claude-Sonnet-3.52026.05 | 65.58 | 66.04 | 68.17 | 62.52 | |
| CIPOBase Model=Seed-Coder-8B2026.05 | 64.99 | 65.33 | 68.82 | 60.82 | |
| GRPOBase Model=Seed-Coder-8B2026.05 | 62.46 | 63.25 | 66.16 | 57.97 | |
| Seed SkillEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 61.5 | — | — | — | |
| TextGradEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 61.5 | — | — | — | |
| ProTeGiEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 61.5 | — | — | — | |
| GEPAEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 61.5 | — | — | — | |
| Qwen2.5-72B-Instruct2026.05 | 61.35 | 61.65 | 61.59 | 60.8 | |
| InitialBase Model=Seed-Coder-8B2026.05 | 60.79 | 61.88 | 63.99 | 56.51 | |
| Qwen3-32B2026.05 | 56.29 | 57.49 | 57 | 54.38 |