Code Generation on Asleep at the Keyboard
93.3Valid RateStarCoderBase
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| StarCoderBaseModel Size=15B2024.02 | 93.3 | 35.6 | |
| StarCoder2Model Size=3B2024.02 | 92.5 | 12.2 | |
| DeepSeekCoderModel Size=6.7B2024.02 | 92.1 | 34.2 | |
| StarCoderBaseModel Size=7B2024.02 | 91.6 | 26.5 | |
| StarCoder2Model Size=7B2024.02 | 91.2 | 39.8 | |
| StarCoderBaseModel Size=3B2024.02 | 91 | 24.6 | |
| CodeLlamaModel Size=13B2024.02 | 90.3 | 30.2 | |
| CodeLlamaModel Size=7B2024.02 | 90 | 21.7 | |
| StarCoder2Model Size=15B2024.02 | 89.8 | 39.2 | |
| DeepSeekCoderModel Size=1.3B2024.02 | 89.3 | 32.5 |