Cross-file code completion on CrossCodeEval TypeScript (test)
70.31Code ESDeepSeekCoder
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DeepSeekCoderParameters=33B2024.02 | 70.31 | 65.14 | |
| DeepSeekCoderParameters=6.7B2024.02 | 69.08 | 63.61 | |
| CodeLlamaParameters=34B2024.02 | 68.98 | 63.19 | |
| CodeLlamaParameters=13B2024.02 | 67.88 | 61.46 | |
| CodeLlamaParameters=7B2024.02 | 65.96 | 59.46 | |
| StableCodeParameters=3B2024.02 | 65.61 | 59.61 | |
| DeepSeekCoderParameters=1.3B2024.02 | 63.59 | 56.41 | |
| StarCoder2Parameters=15B2024.02 | 48.63 | 43.78 | |
| StarCoderBaseParameters=15B2024.02 | 45.24 | 40.47 | |
| StarCoderBaseParameters=7B2024.02 | 44.21 | 39.77 | |
| StarCoder2Parameters=7B2024.02 | 43.19 | 38.84 | |
| StarCoderBaseParameters=3B2024.02 | 41.42 | 35.26 | |
| StarCoder2Parameters=3B2024.02 | 38.79 | 35.17 |