Function-level Code Localization on SWE-bench lite
73.36Acc@5LOCAGENT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LOCAGENTType=Agent-Based, Loc-Model=Claude-3.52025.03 | 73.36 | 77.37 | |
| LOCAGENTType=Agent-Based, Loc-Model=Qwen2.5-32B(ft)2025.03 | 71.9 | 77.01 | |
| OpenhandsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 68.25 | 70.07 | |
| Moatless ToolsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 64.6 | 64.96 | |
| OpenhandsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 64.23 | 64.6 | |
| LOCAGENTType=Agent-Based, Loc-Model=Qwen2.5-7B(ft)2025.03 | 64.23 | 71.53 | |
| AgentlessType=Procedure-Based, Loc-Model=Claude-3.52025.03 | 58.76 | 58.76 | |
| Moatless ToolsType=Agent-Based, Loc-Model=GPT-4o2025.03 | 57.3 | 59.49 | |
| AgentlessType=Procedure-Based, Loc-Model=GPT-4o2025.03 | 55.47 | 55.47 | |
| CodeRankEmbedType=Embedding-Based2025.03 | 51.82 | 58.76 | |
| OpenhandsType=Agent-Based, Loc-Model=GPT-4o2025.03 | 49.64 | 50.36 | |
| SWE-agentType=Agent-Based, Loc-Model=GPT-4o2025.03 | 45.99 | 46.35 | |
| Jina-Code-v2Type=Embedding-Based2025.03 | 42.34 | 52.19 | |
| E5-base-v2Type=Embedding-Based2025.03 | 39.42 | 51.09 | |
| Codesage-large-v2Type=Embedding-Based2025.03 | 33.94 | 44.53 | |
| BM25Type=Embedding-Based2025.03 | 31.75 | 36.86 |