Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

SWE-Gym dataset and benchmarks · SOTA2 Research

Research Datasets

SWE-Gym

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Software Engineering Task Resolution

SWE-Gym (Holdout)

19Task Resolution Rate

12

Failure prediction

SWE-Gym (Transfer held-out)

60ROC-AUC

4

Failure prediction

SWE-Gym (Within corpus)

85.3ROC-AUC

4

Leave-one-corpus-out transfer

SWE-Gym

59.9AUC

3

Popular tasks

Failure Prediction Software Engineering Task Resolution Leave-one-corpus-out transfer

Follow for updates

Join SOTA2 Contact