Software Engineering Issue Resolution on Terminal Bench
32.5Resolve RateHybrid
Evaluation results
| Method | Links | |
|---|---|---|
| HybridFeedback=Hybrid (Execution-free + Execution-based)2025.12 | 32.5 | |
| Execution-free onlyFeedback=Execution-free (SWE-RM)2025.12 | 31.3 | |
| Execution-based onlyFeedback=Execution-based2025.12 | 30 | |
| Poor Calibrated RMFeedback=Ablated RM (Poorly Calibrated)2025.12 | 15 |