Code-Intensive Task Generation on GitHub Repositories
50,137Instances CountSWE-smith
Evaluation results
| Method | Links | |
|---|---|---|
| SWE-smithGold Instance=Pre-installed Github Repo, Data Engine=PRs / LLM Synthesis, Problem Instance=Environment / Failed Unit Tests, Collection=Auto2026.02 | 50,137 | |
| SWE-DevGold Instance=Pre-installed Github Repo, Data Engine=Test-Driven, Problem Instance=Environment / Failed Unit Tests, Collection=Auto2026.02 | 14,000 | |
| R2E-GymGold Instance=Pre-installed Github Repo, Data Engine=PR-based / LLM-based, Problem Instance=Code-Intensive Issue, Collection=Auto2026.02 | 8,135 | |
| SWE-GymGold Instance=Pre-installed Github Repo, Data Engine=PR-based, Problem Instance=Code-Intensive Issue, Collection=Auto2026.02 | 2,438 | |
| SWE-BenchGold Instance=Pre-installed Github Repo, Data Engine=PR-based, Problem Instance=Code-Intensive Issue, Collection=Auto2026.02 | 2,294 |