Parallel Code Generation on ParEval
98Build@1Gemini-2.5-Pro
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini-2.5-ProCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 98 | 77 | 10.4 | |
| Gemini-3-ProCode=Rust, Sched.=Rayon, Temperature=0.22026.03 | 97 | 82 | 7.42 | |
| Qwen2.5-Coder-32BCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 93 | 11 | 9.98 | |
| DeepSeek-6.7B-BaseCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 89 | 11 | 3.65 | |
| DeepSeek-SyntaxCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 85 | 12 | 6.6 | |
| Gemini-2.5-ParlayCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 84 | 33 | 106.87 | |
| Qwen2.5-Coder-32BCode=Rust, Sched.=Rayon, Temperature=0.22026.03 | 82 | 45 | 5.64 | |
| DeepSeek-Coder-V2-Lite-BaseCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 80 | 9 | 2.57 | |
| DeepSeek-ParlayCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 79 | 35 | 16.4 | |
| StarCoder2-15BCode=Rust, Sched.=Rayon, Temperature=0.22026.03 | 77 | 27 | 3.58 | |
| GPT-5 ThinkingCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 73 | 63 | 14.03 | |
| DeepSeek-Coder-V2-Lite-BaseCode=Rust, Sched.=Rayon, Temperature=0.22026.03 | 73 | 29 | 6.26 | |
| Qwen3-RustCode=Rust, Sched.=Rayon, Temperature=0.22026.03 | 64 | 46 | 6.1 | |
| Qwen2.5-Coder-32B-InstructCode=Rust, Sched.=Rayon, Temperature=0.22026.03 | 63 | 49 | 5.97 | |
| Qwen2.5-Coder-32B-InstructCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 61 | 41 | 12.91 | |
| Qwen3-Coder-30B-InstructCode=Rust, Sched.=Rayon, Temperature=0.22026.03 | 61 | 50 | 5.7 | |
| Gemini-2.5-FlashCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 58 | 29 | 13.42 | |
| Qwen3-ParlayCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 50 | 33 | 8.63 | |
| DeepSeek-Coder-V2-Lite-InstructCode=Rust, Sched.=Rayon, Temperature=0.22026.03 | 40 | 2 | 0.77 | |
| Claude Opus 4.5Code=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 28 | 27 | 0.65 | |
| Gemini-3-ProCode=Parlay, Sched.=Parlay, Temperature=0.22026.03 | 25 | 23 | 12.29 |