Code Generation
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
77.5Quality Score
2
Jun 26, 2026
0.01Cost (cents)
2
Jun 26, 2026
0.03Cost (cents)
2
Jun 26, 2026
0.03Cost (cents)
2
Jun 26, 2026
0.02Cost (cents)
2
Jun 26, 2026
0.02Cost (cents)
2
Jun 26, 2026
0.02Cost (cents)
2
Jun 26, 2026
0.33Cost (cents)
2
Jun 26, 2026
0.07Cost (cents)
2
Jun 26, 2026
93.4Quality (%)
2
Jun 26, 2026
29.27Accuracy
2
Jun 23, 2026
11Effect (pp)
2
Jun 23, 2026
38Pass Rate
2
Jun 9, 2026
5Positive Seeds Count
2
Jun 8, 2026
39Accuracy
2
Jun 4, 2026
1.91Throughput
2
May 27, 2026
39.4Accuracy
2
May 20, 2026
45Accuracy
2
May 20, 2026
49.4Accuracy
2
May 20, 2026
90.9pass@1
2
May 19, 2026
82.4pass@1
2
May 19, 2026
64.02Pass@k
2
May 19, 2026
80.49Pass@k
2
May 19, 2026
45.97Pass@k
2
May 19, 2026
32.07Pass@k
2
May 19, 2026