File-level Code Localization on SWE-bench lite
77.74Acc@1LOCAGENT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LOCAGENTType=Agent-Based, Loc-Model=Claude-3.52025.03 | 77.74 | 91.97 | 94.16 | |
| OpenhandsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 77.37 | 87.23 | 90.15 | |
| OpenhandsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 76.28 | 89.78 | 90.15 | |
| LOCAGENTType=Agent-Based, Loc-Model=Qwen2.5-32B(ft)2025.03 | 75.91 | 90.51 | 92.7 | |
| Moatless ToolsType=Agent-Based, Loc-Model=GPT-4o2025.03 | 73.36 | 84.31 | 85.04 | |
| AgentlessType=Procedure-Based, Loc-Model=Claude-3.52025.03 | 72.63 | 79.2 | 79.56 | |
| Moatless ToolsType=Agent-Based, Loc-Model=Claude-3.52025.03 | 72.63 | 85.77 | 86.13 | |
| LOCAGENTType=Agent-Based, Loc-Model=Qwen2.5-7B(ft)2025.03 | 70.8 | 84.67 | 88.32 | |
| AgentlessType=Procedure-Based, Loc-Model=GPT-4o2025.03 | 67.15 | 74.45 | 74.45 | |
| OpenhandsType=Agent-Based, Loc-Model=GPT-4o2025.03 | 60.95 | 71.9 | 73.72 | |
| SWE-agentType=Agent-Based, Loc-Model=GPT-4o2025.03 | 57.3 | 64.96 | 68.98 | |
| CodeRankEmbedType=Embedding-Based2025.03 | 52.55 | 77.74 | 84.67 | |
| E5-base-v2Type=Embedding-Based2025.03 | 49.64 | 74.45 | 80.29 | |
| Codesage-large-v2Type=Embedding-Based2025.03 | 47.81 | 69.34 | 78.1 | |
| Jina-Code-v2Type=Embedding-Based2025.03 | 43.43 | 71.17 | 80.29 | |
| BM25Type=Embedding-Based2025.03 | 38.69 | 51.82 | 61.68 |