Code Generation on MultiPL-E
79.5Average Pass@1Qwen2.5-VL-72B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-VL-72BModel Type=Instruct, Parameters=72B2025.02 | 79.5 | — | — | — | — | — | — | |
| Qwen2.5-72BModel Type=Instruct, Parameters=72B2025.02 | 75.1 | — | — | — | — | — | — | |
| Llama-3.1-405BModel Type=Instruct, Parameters=405B2025.02 | 73.5 | — | — | — | — | — | — | |
| Qwen2-72BModel Type=Instruct, Parameters=72B2025.02 | 69.2 | — | — | — | — | — | — | |
| Llama-3.1-70BModel Type=Instruct, Parameters=70B2025.02 | 68.2 | — | — | — | — | — | — | |
| Qwen2-57B-A14BArchitecture=MoE, # Act Params=14B, # Params=57B2024.07 | 49.8 | — | — | — | — | — | — | |
| WizardCoder-CLParams=34B2024.06 | 47.5 | 44.9 | 55.3 | 47.2 | 47.2 | 44.3 | 46.2 | |
| UNICODERParams=7B2024.06 | 41.6 | 46.4 | 50.2 | 39.2 | 40.4 | 41.2 | 32.4 | |
| CodeLlama-InstructParams=34B2024.06 | 40.5 | 41.5 | 45.9 | 41.5 | 37 | 37.6 | 39.3 | |
| CodeLlamaParams=34B2024.06 | 39.6 | 40.2 | 41.7 | 41.4 | 40.4 | 35.3 | 38.7 | |
| CodeLlama-PythonParams=34B2024.06 | 39.5 | 39.5 | 44.7 | 39.1 | 39.8 | 34.3 | 39.7 | |
| Yi-1.5-34BArchitecture=Dense, # Act Params=32B, # Params=32B2024.07 | 39.5 | — | — | — | — | — | — | |
| Mixtral-8x7BArchitecture=MoE, # Act Params=12B, # Params=47B2024.07 | 39 | — | — | — | — | — | — | |
| Qwen1.5-32BArchitecture=Dense, # Act Params=34B, # Params=34B2024.07 | 38.5 | — | — | — | — | — | — | |
| WizardCoder-SCParams=15B2024.06 | 36.1 | 35.8 | 41.9 | 39 | 39.3 | 33.7 | 27.1 | |
| CodeLlama-PythonParams=7B2024.06 | 29.7 | 29.1 | 35.7 | 30.2 | 29 | 27.1 | 27 | |
| CodeLlamaParams=7B2024.06 | 27.4 | 29.3 | 31.7 | 27 | 25.1 | 25.6 | 25.5 | |
| StarCoderParams=15B2024.06 | 27.2 | 30.2 | 30.8 | 31.6 | 26.1 | 22.7 | 21.8 | |
| StarCoderBaseParams=15B2024.06 | 26.5 | 28.5 | 31.7 | 30.6 | 26.8 | 16.7 | 24.5 | |
| GPT-3.5Params=-2024.06 | — | 69.2 | 67.1 | 63.4 | 60.9 | — | — | |
| GPT-4Params=-2024.06 | — | 81.6 | 78 | 76.4 | 77.2 | — | — |