Code Generation
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
1.753Decode Speed (tokens/s)
6
May 19, 2026
98.8SR
6
May 19, 2026
2.1Acceptance Length
6
May 12, 2026
4.7Acceptance Length
6
May 12, 2026
50Pass@1
6
May 11, 2026
170.9TPS (tok/s)
6
May 27, 2026
42.8Accuracy
6
Apr 29, 2026
21.1Average@4 Score
6
Apr 28, 2026
70.8Success Rate (avg@4)
6
Apr 28, 2026
83.1Pass@4
6
Apr 28, 2026
79.6Pass Rate @4
6
Apr 28, 2026
85.5Pass@4 (HumanEval)
6
Apr 28, 2026
38.8Accuracy
6
Apr 21, 2026
40.85Accuracy
6
Apr 21, 2026
43.9Accuracy
6
Apr 21, 2026
36.5Accuracy
6
Apr 20, 2026
35Accuracy
6
May 27, 2026
1.63Delta HumanEval (pp)
6
Apr 20, 2026
90Beginner Win-rate
6
Apr 16, 2026
49.39Score
6
Apr 13, 2026
48.78Code Generation Score
6
Apr 13, 2026
43.9Score
6
Apr 13, 2026
31.2Score
6
Apr 13, 2026
43.29Score
6
Apr 13, 2026
62pass@1
6
Apr 10, 2026