Code Completion
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
68.26PCR
33
Apr 3, 2026
100HCR
33
Apr 3, 2026
65.6Pass@1
33
Feb 26, 2026
56.7Pass@1
33
Feb 26, 2026
52.05Pass@1
30
May 22, 2026
18.15Edit Distance
30
May 22, 2026
57.7Exact Similarity (ES)
21
Feb 26, 2026
16.2Pass@1
20
Feb 26, 2026
46.1Pass@1
20
Feb 26, 2026
0.652Pass@1
20
Feb 26, 2026
45.8Full Score
17
Feb 26, 2026
79.9MBPP Pass Rate
17
Feb 26, 2026
66.5HE Score
17
Feb 26, 2026
0.731Similarity
17
May 25, 2026
7False Positive Rate (FPR)
16
Apr 14, 2026
7FPR
16
Apr 14, 2026
69.43-Shot Pass Rate
16
Feb 26, 2026
79.1Pass@1
11
Feb 26, 2026
56.1Matplotlib Success Rate
8
Feb 26, 2026
68.3MethodClass Accuracy
7
Feb 26, 2026
52Edit Similarity
6
May 12, 2026
59.78Accuracy
5
May 19, 2026
74Accuracy (HumanEval FIM)
5
Mar 10, 2026
55.2Success Rate (Matplotlib)
5
Feb 26, 2026
9.08C-EM
4
Feb 26, 2026