Issue Resolving on SWE-bench lite
5RoundsMoatlessTools
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| MoatlessToolsLanguage Model=GPT-4o2025.03 | 5 | 0.46 | 1.3 | |
| MoatlessToolsLanguage Model=Claude-3.52025.03 | 5 | 0.46 | 1.4 | |
| LOCAGENTLanguage Model=Qwen2.5-7B(ft)2025.03 | 6 | 0.05 | 13.2 | |
| LOCAGENTLanguage Model=Claude-3.52025.03 | 7 | 0.66 | 1.2 | |
| SWE-agentLanguage Model=GPT-4o2025.03 | 8 | 0.56 | 0.8 | |
| SWE-agentLanguage Model=Claude-3.52025.03 | 9 | 0.67 | 1 | |
| LOCAGENTLanguage Model=Qwen2.5-32B(ft)2025.03 | 9 | 0.09 | 8.6 | |
| OpenhandsLanguage Model=Claude-3.52025.03 | 13 | 0.79 | 0.9 | |
| OpenhandsLanguage Model=GPT-4o2025.03 | 15 | 0.83 | 0.6 |