Code Generation
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
73.78Functional Correctness Score
7
Feb 26, 2026
80.49Score
7
Feb 26, 2026
39Pass@1
7
Feb 26, 2026
0.726Accuracy
7
Feb 26, 2026
50.4Pass@1
7
Feb 26, 2026
39.7Pass@1
7
Feb 26, 2026
2.65Perplexity
7
Feb 26, 2026
88.9Pass@1 Accuracy
6
Jun 30, 2026
87.8HumanEval Score (ID 4)
6
Jun 18, 2026
57Score
6
Jun 11, 2026
52.2Pass Ratio
6
Jun 9, 2026
75.87Accuracy
6
Jun 9, 2026
83.54Accuracy
6
Jun 9, 2026
63.62Pass@1
6
Jun 5, 2026
2.9McCabe Complexity
6
Jun 2, 2026
19.63Metric m Score
6
May 28, 2026
53.5Pass@4
6
May 27, 2026
51.59Accuracy
6
May 27, 2026
86.89Accuracy
6
May 26, 2026
88.89Accuracy
6
May 26, 2026
-0.37Accuracy Degradation (%)
6
May 25, 2026
75HumanEval+ Score
6
Jun 23, 2026
47.56Pass@1
6
May 22, 2026
64.5Execution Match (EM)
6
May 19, 2026
52.4Exact Match (EM)
6
May 19, 2026