Cross-file code completion on CrossCodeEval (test)
49.3Python EMCodeStral-22B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CodeStral-22BModel Size Category=20B+2026.03 | 49.3 | 82.7 | 44.1 | 71.1 | 51 | 85 | 53.7 | 83.6 | 49.5 | 80.6 | |
| Qwen2.5-Coder-32BModel Size Category=20B+2026.03 | 49.2 | 82.1 | 56.4 | 86.6 | 54.9 | 87 | 68 | 91.6 | 57.1 | 86.8 | |
| IQuest-Coder-V1-40BModel Size Category=20B+2026.03 | 49 | 81.7 | 57.9 | 86.2 | 61.9 | 88.5 | 63.4 | 85.5 | 57.8 | 85.7 | |
| Qwen2.5-Coder-14BModel Size Category=14B+2026.03 | 47.7 | 81.7 | 54.7 | 85.7 | 52.9 | 86 | 66.4 | 91.1 | 55.4 | 86.1 | |
| DS-Coder-33B-BaseModel Size Category=20B+2026.03 | 44.2 | 80.4 | 46.5 | 82.7 | 49.2 | 84 | 55.2 | 87.8 | 48.8 | 83.7 | |
| Qwen2.5-Coder-7BModel Size Category=6B+2026.03 | 42.4 | 78.6 | 48.1 | 82.6 | 46.8 | 83.4 | 59.7 | 87.9 | 49.3 | 83.1 | |
| DS-Coder-V2-Lite-BaseModel Size Category=6B+2026.03 | 41.8 | 78.3 | 46.1 | 81.2 | 44.6 | 81.4 | 58.7 | 87.9 | 47.8 | 82.2 | |
| DeepSeek-Coder-6.7B-BaseModel Size Category=6B+2026.03 | 41.1 | 79.2 | 39.9 | 80.1 | 46.3 | 82.4 | 55 | 86.9 | 45.6 | 82.1 | |
| CodeQwen1.5-7BModel Size Category=6B+2026.03 | 40.7 | 77.8 | 47 | 81.6 | 45.8 | 82.2 | 59.7 | 87.6 | 48.3 | 82.3 | |
| StarCoder2-15BModel Size Category=14B+2026.03 | 28.2 | 70.5 | 26.7 | 71 | 24.7 | 76.3 | 25.2 | 74.2 | 26.2 | 73 | |
| DeepSeek-Coder-BaseSize=6.7B, Retrieval=true, Repo Pre-training=true2024.01 | 16.14 | 66.51 | 17.72 | 63.18 | 14.03 | 61.77 | 16.23 | 63.42 | — | — | |
| DeepSeek-Coder-BaseSize=6.7B, Retrieval=true, Repo Pre-training=false2024.01 | 16.02 | 66.65 | 16.64 | 61.88 | 13.23 | 60.92 | 14.48 | 62.38 | — | — | |
| StarCoder-BaseSize=7B, Retrieval=true2024.01 | 13.06 | 64.24 | 15.61 | 64.78 | 7.54 | 42.06 | 14.2 | 65.03 | — | — | |
| CodeLlama-BaseSize=7B, Retrieval=true2024.01 | 13.02 | 64.3 | 16.41 | 64.64 | 12.34 | 60.64 | 13.19 | 63.04 | — | — | |
| StarCoder2-7BModel Size Category=6B+2026.03 | 10.9 | 63.1 | 8.3 | 71 | 6.7 | 76.8 | 7.3 | 72.1 | 8.3 | 70.8 | |
| CodeGeex2Size=6B, Retrieval=true2024.01 | 10.73 | 61.76 | 10.1 | 59.56 | 7.72 | 55.17 | 4.64 | 52.3 | — | — | |
| DeepSeek-Coder-BaseSize=6.7B, Retrieval=false, Repo Pre-training=true2024.01 | 9.53 | 61.65 | 10.8 | 61.77 | 9.59 | 60.17 | 5.26 | 61.32 | — | — | |
| CodeGeex2Size=6B, Retrieval=false2024.01 | 8.11 | 59.55 | 7.34 | 59.6 | 6.14 | 55.5 | 1.7 | 51.66 | — | — | |
| CodeLlama-BaseSize=7B, Retrieval=false2024.01 | 7.32 | 59.66 | 9.68 | 62.64 | 8.19 | 58.5 | 4.07 | 59.19 | — | — | |
| StarCoder-BaseSize=7B, Retrieval=false2024.01 | 6.68 | 59.55 | 8.65 | 62.57 | 5.01 | 48.83 | 4.75 | 59.53 | — | — |