Code Generation on BigCodeBench (mean accuracy)
46.7Mean AccuracyConsensus-WUCS
Evaluation results
| Method | Links | |
|---|---|---|
| Consensus-WUCSModel=Qwen2.5-32B-Coder2025.02 | 46.7 | |
| CodeRSAModel=Qwen2.5-32B-Coder2025.02 | 45.4 | |
| CoderModel=Qwen2.5-32B-Coder2025.02 | 45 | |
| CoderReviewerModel=Qwen2.5-32B-Coder2025.02 | 44.9 | |
| RandomModel=Qwen2.5-32B-Coder2025.02 | 41.8 | |
| CodeRSAModel=Llama-3-70B2025.02 | 39.5 | |
| CoderReviewerModel=Llama-3-70B2025.02 | 37.6 | |
| CoderModel=Llama-3-70B2025.02 | 37.1 | |
| RandomModel=Llama-3-70B2025.02 | 36.9 | |
| Consensus-WUCSModel=Llama-3-70B2025.02 | 36.1 | |
| CodeRSAModel=Qwen2.5-7B2025.02 | 34 | |
| CoderReviewerModel=Qwen2.5-7B2025.02 | 31.9 | |
| RandomModel=Qwen2.5-7B2025.02 | 31.3 | |
| CoderModel=Qwen2.5-7B2025.02 | 30.7 | |
| Consensus-WUCSModel=Qwen2.5-7B2025.02 | 29.7 | |
| CodeRSAModel=Llama-3-8B2025.02 | 20.2 | |
| Consensus-WUCSModel=Llama-3-8B2025.02 | 17.1 | |
| RandomModel=Llama-3-8B2025.02 | 15.1 | |
| CoderReviewerModel=Llama-3-8B2025.02 | 12.8 | |
| CoderModel=Llama-3-8B2025.02 | 10.1 |