Code Generation
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
55.2MBPP Accuracy (Pre-Exec)
4
May 26, 2026
46.8Base Pass Rate
4
May 25, 2026
42.1Accuracy (Base)
4
May 25, 2026
96.72Mean F1 Score
4
May 21, 2026
6.61MAT Score
4
May 20, 2026
2.82Success Rate (SR)
4
May 20, 2026
4.49SGLang Acceptance Length
4
May 12, 2026
5.21Acceptance Length
4
May 12, 2026
7.8Avg@4
4
May 12, 2026
92.2Accuracy
4
May 11, 2026
71.5Pass@1
4
May 11, 2026
193.1Throughput (tok/s)
4
May 7, 2026
91.5Pass@1
4
May 7, 2026
41.4LCB Score
4
May 6, 2026
84.1HumanEval+
4
May 6, 2026
100Accuracy (Attack)
4
Apr 28, 2026
1MAT
4
Apr 24, 2026
33.71Accuracy
4
Apr 16, 2026
38Accuracy
4
Apr 13, 2026
63Accuracy
4
Apr 13, 2026
54.6Accuracy
4
Apr 10, 2026
55.5Accuracy
4
Apr 10, 2026
68.1CodeBS
4
Apr 3, 2026
70.2MBPP Score
4
Apr 1, 2026
77.44HE Score
4
Apr 1, 2026