Code Localization on SWE-bench Verified (test)
86.38File PrecisionFuseSearch*
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| FuseSearch*Model=Claude Haiku 4.5, Mode=Sequential, Note=Sequential prompts mode2026.01 | 86.38 | 62.44 | 72.48 | 66.3 | 47.45 | 55.31 | 23.9 | 90.3 | 270 | |
| FuseSearch (train)Model=Qwen3-4B-Instruct, Mode=Parallel2026.01 | 83.59 | 85.75 | 84.65 | 59.91 | 53.33 | 56.43 | 4.78 | 5.43 | 30.9 | |
| FuseSearch (train)Model=Qwen3-30B-A3B-Instruct, Mode=Parallel2026.01 | 83.12 | 82.9 | 83.01 | 66.58 | 52.35 | 58.62 | 5.77 | 10.6 | 43.2 | |
| FuseSearch*Model=Kimi-K2-Instruct, Mode=Sequential, Note=Sequential prompts mode2026.01 | 77.33 | 79.53 | 78.42 | 51.87 | 49.02 | 50.4 | 15 | 71.3 | 216 | |
| FuseSearchModel=Kimi-K2-Instruct, Mode=Parallel2026.01 | 75.11 | 89.31 | 81.6 | 51 | 54.9 | 52.88 | 7.92 | 43.6 | 62.1 | |
| FuseSearchModel=Claude Haiku 4.5, Mode=Parallel2026.01 | 73.54 | 94.5 | 82.71 | 48.58 | 70.61 | 57.56 | 6.24 | 36.2 | 110 | |
| FuseSearch (base)Model=Qwen3-30B-A3B-Instruct, Mode=Parallel2026.01 | 70.41 | 79.53 | 74.7 | 53.27 | 46.27 | 45.65 | 7.5 | 14.9 | 80.1 | |
| FuseSearch (base)Model=Qwen3-4B-Instruct, Mode=Parallel2026.01 | 64.75 | 64.25 | 64.5 | 43.95 | 34.9 | 38.91 | 4.24 | 6.12 | 47.9 | |
| LocAgentModel=Claude Haiku 4.5, Mode=Sequential2026.01 | 61.57 | 87.56 | 72.3 | 41.29 | 65.49 | 50.64 | 17.3 | 318 | 567 | |
| LocAgentModel=Kimi-K2-Instruct, Mode=Sequential2026.01 | 55.39 | 95.85 | 70.21 | 33.1 | 73.33 | 45.61 | 14.9 | 261 | 447 | |
| LocAgentModel=Qwen3-30B-A3B-Instruct, Mode=Sequential2026.01 | 45.16 | 62.95 | 52.59 | 29.96 | 32.75 | 31.32 | 11.4 | 112 | 136 | |
| RepoSearcherModel=Qwen3-30B-A3B-Instruct, Mode=Sequential2026.01 | 40.25 | 50.78 | 44.9 | 20.07 | 34.71 | 25.43 | 16.7 | 92.4 | 113 | |
| LocAgentModel=Qwen3-4B-Instruct, Mode=Sequential2026.01 | 39.1 | 56.22 | 46.12 | 27.37 | 31.18 | 26.97 | 6.09 | 109 | 135 | |
| AgentlessModel=Claude Haiku 4.5, Mode=Sequential2026.01 | 38.82 | 91.71 | 54.55 | 21.48 | 61.37 | 31.83 | 2 | 7.32 | 10.6 | |
| AgentlessModel=Kimi-K2-Instruct, Mode=Sequential2026.01 | 34.53 | 92.23 | 50.25 | 28.22 | 56.27 | 37.59 | 2 | 11.8 | 8.33 | |
| RepoSearcherModel=Qwen3-4B-Instruct, Mode=Sequential2026.01 | 31.99 | 47.15 | 38.12 | 16.92 | 30.39 | 21.74 | 14.8 | 85.3 | 99.2 | |
| AgentlessModel=Qwen3-4B-Instruct, Mode=Sequential2026.01 | 28.11 | 76.68 | 41.14 | 11.62 | 34.12 | 17.33 | 2 | 4.24 | 8.33 | |
| AgentlessModel=Qwen3-30B-A3B-Instruct, Mode=Sequential2026.01 | 24.22 | 87.56 | 38.19 | 15.34 | 55.1 | 24 | 2 | 8.04 | 32 | |
| CoSILModel=Kimi-K2-Instruct, Mode=Sequential2026.01 | 21.33 | 95.6 | 34.88 | 23.04 | 70.2 | 34.69 | 6.8 | 58.8 | 94 | |
| CoSILModel=Qwen3-30B-A3B-Instruct, Mode=Sequential2026.01 | 21.22 | 96.11 | 34.77 | 18.4 | 66.86 | 28.86 | 11.1 | 49.2 | 66.8 | |
| CoSILModel=Qwen3-4B-Instruct, Mode=Sequential2026.01 | 21.21 | 94.82 | 34.66 | 18.17 | 65.49 | 28.45 | 10.8 | 50.8 | 63.9 | |
| RepoSearcherModel=Kimi-K2-Instruct, Mode=Sequential2026.01 | 20.61 | 96.11 | 33.94 | 25.2 | 72.35 | 37.39 | 15.6 | 94.5 | 108 | |
| CoSILModel=Claude Haiku 4.5, Mode=Sequential2026.01 | 19.62 | 96.63 | 32.62 | 18.37 | 69.41 | 29.05 | 7.22 | 38.7 | 53 | |
| RepoSearcherModel=Claude Haiku 4.5, Mode=Sequential2026.01 | 19.55 | 97.41 | 32.57 | 20.33 | 67.65 | 31.26 | 19.7 | 114 | 193 |