Software Engineering Task Resolution on SWE-Gym (Holdout)
19Task Resolution RateDAgger-style
Evaluation Results
| Method | Links | |
|---|---|---|
| DAgger-styleBackbone=Qwen3-8B, Scaffold=OpenHands, Data=SWE-Gym2026.05 | 19 | |
| DAgger-styleBackbone=Qwen3-4B, Scaffold=OpenHands, Data=SWE-Gym2026.05 | 17 | |
| AggreVaTe-styleBackbone=Qwen3-8B, Scaffold=OpenHands, Data=SWE-Gym2026.05 | 17 | |
| Qwen3-4B + OPDBackbone=Qwen3-4B, Scaffold=OpenHands, Data=SWE-Gym2026.05 | 16 | |
| AggreVaTe-styleBackbone=Qwen3-4B, Scaffold=OpenHands, Data=SWE-Gym2026.05 | 16 | |
| Qwen3-8B + OPDBackbone=Qwen3-8B, Scaffold=OpenHands, Data=SWE-Gym2026.05 | 16 | |
| Qwen3-4B + SFTBackbone=Qwen3-4B, Scaffold=OpenHands, Data=SWE-Gym2026.05 | 15 | |
| Qwen3-8B + SFTBackbone=Qwen3-8B, Scaffold=OpenHands, Data=SWE-Gym2026.05 | 12 | |
| Qwen3-4B + GRPOBackbone=Qwen3-4B, Scaffold=OpenHands, Data=SkyRL-v02026.05 | 8 | |
| Qwen3-4BBackbone=Qwen3-4B, Scaffold=OpenHands, Data=–2026.05 | 5 | |
| Qwen3-8B + GRPOBackbone=Qwen3-8B, Scaffold=OpenHands, Data=SkyRL-v02026.05 | 4 | |
| Qwen3-8BBackbone=Qwen3-8B, Scaffold=OpenHands, Data=–2026.05 | 2 |