Code Localization on LocBench V1
8Mean Time (s)BM25
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BM252026.05 | 8 | 5.2 | — | — | — | |
| mini-swe-agentModel=GPT-5.22026.05 | 33.5 | 25.4 | — | — | — | |
| AgentlessModel=GPT-5.22026.05 | 48.5 | 34.7 | — | — | — | |
| LARGERModel=GPT-5.22026.05 | 60 | 40.9 | — | — | — | |
| Claude CodeInterface=Native2026.05 | 69.4 | 54.4 | — | — | — | |
| SWE-agentModel=GPT-5.22026.05 | 79.9 | 71.5 | — | — | — | |
| CodexInterface=Native2026.05 | 129.2 | 122.4 | — | — | — | |
| CoSILModel=GPT-5.22026.05 | 151.9 | 98.8 | — | — | — | |
| LocAgentModel=GPT-5.22026.05 | 623.6 | 167.6 | — | — | — | |
| OpenHandsInterface=Native2026.05 | 710.7 | 483.1 | — | — | — | |
| AgentlessModel=GPT-5.22026.05 | — | — | 26,300 | — | 0.05 | |
| CodexInterface=native2026.05 | — | — | 591,400 | — | 1.1 | |
| CoSILModel=GPT-5.22026.05 | — | — | 25,000 | — | 0.05 | |
| LARGERModel=GPT-5.22026.05 | — | — | 452,000 | 14.5 | 0.56 | |
| LocAgentModel=GPT-5.22026.05 | — | — | 940,700 | — | 1.67 | |
| mini-swe-agentModel=GPT-5.22026.05 | — | — | — | 16.3 | 0.1 | |
| OpenHandsModel=GPT-5.22026.05 | — | — | — | 23.6 | — | |
| SWE-agentModel=GPT-5.22026.05 | — | — | 105,000 | 15.3 | 0.14 |