Code Generation on LiveCodeBench May 2023 to February 2024 (start date 2023-07-01)
24.1Pass@1CodeQwen1.5-7B-Chat
Evaluation Results
| Method | Links | |
|---|---|---|
| CodeQwen1.5-7B-ChatParameters=7B, Variant=Chat2024.10 | 24.1 | |
| SelfCodeAlign-CQ-7BParameters=7B, Variant=CQ2024.10 | 22.8 | |
| CodeQwen1.5-7B-OctoPackParameters=7B, Training Technique=OctoPack2024.10 | 21.8 | |
| DeepSeek-Coder-6.7B-InstructParameters=6.7B, Variant=Instruct2024.10 | 20.8 | |
| CodeQwen1.5-7B-BaseParameters=7B, Variant=Base2024.10 | 20.7 | |
| Llama-3-8B-InstructParameters=8B, Variant=Instruct2024.10 | 18.4 | |
| StarCoder2-15BParameters=15B2024.10 | 14.7 | |
| CodeGemma-7B-ITParameters=7B, Variant=IT2024.10 | 14.1 | |
| OctoCoder-16BParameters=16B2024.10 | 11.2 |