Code Generation
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
39.2Score
2
May 15, 2026
34.76Pass@1 Accuracy
2
May 13, 2026
8.98AC Rate
2
May 8, 2026
9.64AC Rate
2
May 8, 2026
8.7Acceptance Rate (AC)
2
May 8, 2026
89Accuracy
2
May 4, 2026
98.38Strict ASR
2
May 1, 2026
38NFE (Number of Failed Executions)
2
Apr 23, 2026
81.8NFE
2
Apr 23, 2026
71.28Pass Rate Before
2
Apr 20, 2026
82.32Pass Rate Before
2
Apr 20, 2026
68.01Pass Rate Before
2
Apr 20, 2026
27.01Pass@1
2
Apr 15, 2026
75.9Accuracy
2
Apr 10, 2026
69.8Accuracy
2
Mar 17, 2026
51.22Final Accuracy
2
Mar 4, 2026
93Functional Score (%)
2
Feb 26, 2026
0.58Inference Cost (USD)
2
Feb 26, 2026
0.2Inference Cost (USD)
2
Feb 26, 2026
0.007MAE
2
Feb 26, 2026
0.017MAE
2
Feb 26, 2026
48.7Pass@1
2
Feb 26, 2026
720Throughput (tok/s)
2
Feb 26, 2026
31.7Pass@1
2
Feb 26, 2026
65.76Score
2
Feb 26, 2026