Code Editing on EditBench Core set
66.67Pass@1Claude Sonnet 4
Evaluation results
| Method | Links | |
|---|---|---|
| Claude Sonnet 4Model Weight Type=Closed Weight2026.03 | 66.67 | |
| GPT o4-miniModel Weight Type=Closed Weight2026.03 | 57.41 | |
| SEMREP QwQ-32BModel Weight Type=Open Weight, Model Size=32B2026.03 | 57.41 | |
| Qwen3-CoderModel Weight Type=Open Weight, Model Size=405B2026.03 | 55.56 | |
| GLM-4.6Model Weight Type=Open Weight, Model Size=355B2026.03 | 55.56 | |
| Gemini 2.5 ProModel Weight Type=Closed Weight2026.03 | 54.63 | |
| Finetuned QwQ-32BModel Weight Type=Open Weight, Model Size=32B2026.03 | 53.7 | |
| Qwen2.5-72B-InstructModel Weight Type=Open Weight, Model Size=72B2026.03 | 53.7 | |
| QwQ-32BModel Weight Type=Open Weight, Model Size=32B2026.03 | 50.93 | |
| SEMREP Qwen2.5-Coder-7BModel Weight Type=Open Weight, Model Size=7B2026.03 | 32.41 | |
| gemma-3-12b-itModel Weight Type=Open Weight, Model Size=12B2026.03 | 23.15 | |
| Finetuned Qwen2.5-Coder-7BModel Weight Type=Open Weight, Model Size=7B2026.03 | 23.15 | |
| Qwen2.5-Coder-7BModel Weight Type=Open Weight, Model Size=7B2026.03 | 18.52 |