Code Cloze on CodeXGLUE max min subtask
87.9Python ScoreInCoder (Left-to-right reranking)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| InCoder (Left-to-right reranking)ordering=left-to-right, context=left context, completion, and right context, scoring=reranking2022.04 | 87.9 | 90.1 | 76.3 | 92.8 | 91.7 | 90.4 | |
| InCoder (CM infill-region)mode=causal masking scoring, masked region=left prefix and 10 tokens from right side context2022.04 | 86.2 | 91.2 | 78.9 | 94.7 | 89.8 | 91.4 | |
| CodeBERT2022.04 | 82.2 | 86.4 | 86.8 | 90.8 | 90.5 | 88.2 | |
| InCoder (CM infill-token)mode=causal masking scoring, masked region=single token2022.04 | 81.8 | 73.9 | 81.6 | 95.4 | 77.6 | 87 | |
| InCoder (Left-to-right single)ordering=left-to-right, context=left context only, scoring=completion (max/min)2022.04 | 76.9 | 77.6 | 65.8 | 70.4 | 74.1 | 77.1 |