Multilingual Code Generation on MultiPL-E (pass@1)
91.2Pass@1 ScoreGemini-3-Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3-Pro2026.06 | 91.2 | |
| Claude-Opus-4.52026.06 | 91 | |
| Qwen3-235B-A22B-Instruct-25072026.06 | 87.9 | |
| Qwen3-Coder-480B-A35B-Instruct2026.06 | 87.4 | |
| GPT-5.12026.06 | 86.2 | |
| DeepSeek-V3.22026.06 | 85.8 | |
| Kimi-K2-Instruct-09052026.06 | 85.7 | |
| Kimi-Dev-72B2026.06 | 80.3 | |
| Qwen2.5-Coder-32B-Instruct2026.06 | 79.6 | |
| Qwen2.5-Coder-14B-Instruct2026.06 | 78.8 | |
| Qwen2.5-Coder-7B-Instruct2026.06 | 75.4 | |
| Seed-Coder-8B-Instruct2026.06 | 75.1 | |
| LoopCoder-v2R (number of refinement loops)=22026.06 | 73.9 | |
| DeepSeek-Coder-V2-Lite-Instruct2026.06 | 71.5 | |
| LoopCoder-v2R (number of refinement loops)=32026.06 | 69.8 | |
| LoopCoder-v2R (number of refinement loops)=12026.06 | 69.5 | |
| GLM-4.72026.06 | 69 | |
| LoopCoder-v2R (number of refinement loops)=42026.06 | 67.3 |