Code Generation
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
97Execution Accuracy
14
Feb 26, 2026
9.48MAT
14
Feb 26, 2026
0.658Execution Accuracy (Simple)
14
Feb 27, 2026
0.7Execution Accuracy (Simple)
14
Feb 27, 2026
0.41Full Score
14
Feb 26, 2026
83.5MBPP Score
14
Feb 26, 2026
88.4HE Pass Rate
14
Feb 26, 2026
89.2Compilation Success Rate
13
May 19, 2026
94.5Compilation Success Rate
13
May 19, 2026
85.2HumanEval+ Score
13
May 6, 2026
27.8Pass@3
13
Apr 16, 2026
78Accuracy (%)
13
Apr 14, 2026
145.99Throughput (tokens/s)
13
Mar 12, 2026
81.94Accuracy
13
Feb 26, 2026
52.4Accuracy
13
Feb 26, 2026
59.1Accuracy
13
Feb 26, 2026
0.94Pass@1
13
Feb 26, 2026
73Acc
13
Feb 26, 2026
5.26Accepted Length (τ)
12
Jul 7, 2026
94.2ROCG
12
Jul 7, 2026
83.3Correctness (pass@1)
12
Jun 30, 2026
80.49Accuracy
12
Jun 30, 2026
75.66Accuracy
12
Jun 30, 2026
95.1Accuracy
12
Jun 23, 2026
11,073Response Length (Tokens)
12
Jun 17, 2026