SWE-Bench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SWE-Bench Sympy Verified (test)
51.11Accuracy
3
SWE-Bench Django Verified (test)
52.23Accuracy
3
SWE-Bench Pro
32.7Score
2
SWE-bench
73.41AS
2
SWE-bench Lite (test)
57.333Resolution Rate
2
SWE-Bench Lite
57.12Accuracy
2
SWE-bench Multimodal
85.39Accuracy
1
SWE-bench Multimodal
2.23MAE
1
SWE-bench Verified
78.54Accuracy
1
SWE-bench Verified
2.09MAE
1