Module-level Code Localization on SWE-bench lite
86.5Acc@5LOCAGENT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LOCAGENTType=Agent-Based, Loc-Model=Claude-3.52025.03 | 86.5 | 87.59 | |
| LOCAGENTType=Agent-Based, Loc-Model=Qwen2.5-32B(ft)2025.03 | 85.77 | 87.23 | |
| OpenhandsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 83.21 | 83.58 | |
| LOCAGENTType=Agent-Based, Loc-Model=Qwen2.5-7B(ft)2025.03 | 81.02 | 82.85 | |
| OpenhandsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 77.74 | 78.1 | |
| Moatless ToolsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 76.28 | 76.28 | |
| Moatless ToolsType=Agent-Based, Loc-Model=GPT-4o2025.03 | 74.82 | 76.28 | |
| CodeRankEmbedType=Embedding-Based2025.03 | 71.9 | 78.83 | |
| AgentlessType=Procedure-Based, Loc-Model=Claude-3.52025.03 | 68.98 | 68.98 | |
| E5-base-v2Type=Embedding-Based2025.03 | 67.88 | 72.26 | |
| AgentlessType=Procedure-Based, Loc-Model=GPT-4o2025.03 | 67.15 | 67.15 | |
| Jina-Code-v2Type=Embedding-Based2025.03 | 63.5 | 72.63 | |
| OpenhandsType=Agent-Based, Loc-Model=GPT-4o2025.03 | 62.41 | 63.87 | |
| Codesage-large-v2Type=Embedding-Based2025.03 | 60.58 | 69.71 | |
| SWE-agentType=Agent-Based, Loc-Model=GPT-4o2025.03 | 58.03 | 58.03 | |
| BM25Type=Embedding-Based2025.03 | 45.26 | 52.92 |