Behavioral Rule-Collision Resolution on WIRE candidate pairs post-generation support G
13,335GAll
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| AllDescription=Aggregated results across all policies2026.05 | 13,335 | 35.4 | 17.7 | 24.4 | 22.5 | 64.6 | -6.7 | |
| Lovable2026.05 | 4,719 | 23.3 | 16.4 | 25.4 | 34.9 | 76.7 | -9 | |
| OpenHands2026.05 | 4,659 | 52.3 | 10.9 | 23.4 | 13.4 | 47.7 | -12.5 | |
| Manus2026.05 | 2,616 | 25.6 | 29.4 | 22.8 | 22.2 | 74.4 | 6.6 | |
| ChatGPT-52026.05 | 732 | 32.8 | 9.3 | 41.7 | 16.3 | 67.2 | -32.4 | |
| Gemini-D2026.05 | 526 | 45.8 | 45.8 | 5.3 | 3 | 54.2 | 40.5 | |
| mini-SWE2026.05 | 83 | 37.3 | 7.2 | 37.3 | 18.1 | 62.7 | -30.1 |