ResearchDatasetsSWE-agentSWE-agentFollowBenchmarksTaskDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyTaskDatasetSOTA resultTrendResultsFailure predictionSWE-agent (Transfer held-out)0.69ROC-AUC7Failure predictionSWE-agent (Within corpus)0.713ROC-AUC4Software engineering task resolutionSWE-agent task distribution74,000Tokens per Success3