Code Merge Conflict Resolution on Merge-Bench All languages
47.1Equivalent Text RateGemini 2.5 Pro
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini 2.5 Pro2026.05 | 47.1 | 52.6 | 42.1 | 5.3 | 0 | |
| Claude Opus 42026.05 | 40.3 | 44.8 | 34.5 | 20.4 | 0.3 | |
| o3 Pro2026.05 | 39.2 | 45.1 | 40.9 | 14.1 | 0 | |
| R1-0528 671BModel size=671B2026.05 | 32 | 36.5 | 26.3 | 36.9 | 0.4 | |
| Grok 42026.05 | 27.7 | 31.7 | 20.9 | 47.3 | 0.1 | |
| Qwen3 235BModel size=235B2026.05 | 25.8 | 30.6 | 32 | 37.3 | 0.1 |