Software Engineering on SWE-bench Multilingual (Pass@k)
21Pass@1SE-agent-Reflect
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SE-agent-ReflectModel=Qwen-72b2026.05 | 21 | 42.3 | 46.6 | |
| HHDModel=Qwen-72b2026.05 | 21 | 44.3 | 48.3 | |
| Agent-RLVR-PlanModel=Qwen-72b2026.05 | 20.4 | 41.6 | 47.3 | |
| Naive RFTModel=Qwen-72b2026.05 | 18.3 | 42.3 | 47.7 | |
| Agent-RLVR-EFModel=Qwen-72b2026.05 | 18.3 | 41.6 | 46.7 | |
| HHDModel=GLM-Air2026.05 | 18.3 | 33 | 39 | |
| Dense Expert JudgeModel=Qwen-72b2026.05 | 18 | 41 | 46.6 | |
| SE-agent-ReflectModel=GLM-Air2026.05 | 18 | 30.6 | 37.3 | |
| HHD-MModel=Qwen-72b2026.05 | 17.6 | 41 | 45.6 | |
| HHD-MModel=GLM-Air2026.05 | 17.3 | 35 | 39 | |
| Agent-RLVR-EFModel=GLM-Air2026.05 | 17 | 30.6 | 34 | |
| Dense Expert JudgeModel=GLM-Air2026.05 | 17 | 32.3 | 38.3 | |
| Naive RFTModel=GLM-Air2026.05 | 16.7 | 30.3 | 37.3 | |
| Agent-RLVR-PlanModel=GLM-Air2026.05 | 16 | 31.3 | 36.3 |