Software Engineering on SWE-bench verified (All)
93.8Success RateACE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACERepo=scikit-learn, N=32, Base Model=GPT-5.42026.04 | 93.8 | 451 | |
| TF-GRPORepo=scikit-learn, N=32, Base Model=GPT-5.42026.04 | 93.8 | 378 | |
| NexAU0Repo=scikit-learn, N=32, Base Model=GPT-5.42026.04 | 93.8 | 307 | |
| AHERepo=scikit-learn, N=32, Base Model=GPT-5.42026.04 | 87.5 | 257 | |
| AHERepo=django, N=231, Base Model=GPT-5.42026.04 | 81 | 484 | |
| ACERepo=django, N=231, Base Model=GPT-5.42026.04 | 79.2 | 707 | |
| NexAU0Repo=django, N=231, Base Model=GPT-5.42026.04 | 79.2 | 527 | |
| TF-GRPORepo=django, N=231, Base Model=GPT-5.42026.04 | 78.8 | 583 | |
| ACERepo=pydata, N=22, Base Model=GPT-5.42026.04 | 77.3 | 563 | |
| TF-GRPORepo=pydata, N=22, Base Model=GPT-5.42026.04 | 77.3 | 516 | |
| NexAU0Repo=pydata, N=22, Base Model=GPT-5.42026.04 | 77.3 | 386 | |
| AHERepo=All, N=500, Base Model=GPT-5.4, Evolution Source=terminal-bench-long-time2026.04 | 75.6 | 461 | |
| NexAU0Repo=All, N=500, Base Model=GPT-5.4, Self-evolution loop=None (Seed)2026.04 | 75.2 | 526 | |
| ACERepo=All, N=500, Base Model=GPT-5.4, Evolution Source=terminal-bench-long-time2026.04 | 74.6 | 679 | |
| TF-GRPORepo=All, N=500, Base Model=GPT-5.4, Evolution Source=terminal-bench-long-time2026.04 | 74.2 | 582 | |
| NexAU0Repo=matplotlib, N=34, Base Model=GPT-5.42026.04 | 73.5 | 486 | |
| AHERepo=matplotlib, N=34, Base Model=GPT-5.42026.04 | 73.5 | 391 | |
| AHERepo=pydata, N=22, Base Model=GPT-5.42026.04 | 72.7 | 338 | |
| NexAU0Repo=sympy, N=75, Base Model=GPT-5.42026.04 | 70.7 | 494 | |
| AHERepo=sympy, N=75, Base Model=GPT-5.42026.04 | 70.7 | 479 | |
| ACERepo=matplotlib, N=34, Base Model=GPT-5.42026.04 | 70.6 | 622 | |
| TF-GRPORepo=matplotlib, N=34, Base Model=GPT-5.42026.04 | 70.6 | 530 | |
| AHERepo=sphinx-doc, N=44, Base Model=GPT-5.42026.04 | 70.5 | 656 | |
| ACERepo=sympy, N=75, Base Model=GPT-5.42026.04 | 69.3 | 602 | |
| NexAU0Repo=sphinx-doc, N=44, Base Model=GPT-5.42026.04 | 68.2 | 731 | |
| TF-GRPORepo=sympy, N=75, Base Model=GPT-5.42026.04 | 68 | 572 | |
| TF-GRPORepo=sphinx-doc, N=44, Base Model=GPT-5.42026.04 | 65.9 | 848 | |
| ACERepo=sphinx-doc, N=44, Base Model=GPT-5.42026.04 | 61.4 | 990 | |
| ACERepo=astropy, N=22, Base Model=GPT-5.42026.04 | 59.1 | 546 | |
| TF-GRPORepo=astropy, N=22, Base Model=GPT-5.42026.04 | 59.1 | 470 | |
| NexAU0Repo=astropy, N=22, Base Model=GPT-5.42026.04 | 54.5 | 667 | |
| AHERepo=astropy, N=22, Base Model=GPT-5.42026.04 | 50 | 277 |