Software Engineering
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
80.9Success Rate
53
Jun 17, 2026
62.6Accuracy
43
Jul 9, 2026
4.66Speedup
36
May 29, 2026
93.8Success Rate
32
Apr 29, 2026
83.8Resolution Rate
32
Jun 1, 2026
10,538Throughput (tok/s)
30
May 29, 2026
88.2Score
24
May 21, 2026
57.09Accuracy
23
Jun 12, 2026
72Pass@1
23
Jul 7, 2026
72Pass Rate
20
May 28, 2026
75.2Resolution Rate
20
Mar 4, 2026
64.3Resolve Rate (Pass@1)
19
Jun 12, 2026
84Pass@1
18
Apr 16, 2026
77.2Resolution Rate
17
Feb 26, 2026
82.4Resolve Rate
16
May 7, 2026
78.2Score
15
May 21, 2026
21Pass@1
14
May 13, 2026
51.6Resolved Rate
14
Mar 4, 2026
72Accuracy
12
May 4, 2026
86.2Resolution Rate (%)
10
May 19, 2026
38.16Avg@8 BC%
9
Jun 4, 2026
31.49Avg@8 BC%
9
Jun 4, 2026
66.8Score
9
Mar 24, 2026
0Misalignment Rate
8
May 28, 2026
52.9Resolved Rate
8
Feb 26, 2026