Trajectory Verification on SWE-bench OpenHands-LM-32B trajectories (Verified)
48.8RM@32SWE-RM-30A3B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SWE-RM-30A3BType=EF (execution-free)2025.12 | 48.8 | 0.748 | 0.08 | |
| DEEP SWEType=EF (execution-free)2025.12 | 44.6 | 0.732 | 0.118 | |
| DEEP SWEType=EB (execution-based)2025.12 | 44.2 | — | — | |
| AGENTLESSType=EB (execution-based)2025.12 | 42.4 | — | — | |
| SWE-GYMType=EF (execution-free)2025.12 | 41.6 | 0.718 | 0.164 |