Code Generation on MBPP+ (Mean Accuracy)
83.7Mean AccuracyConsensus-WUCS
Evaluation results
| Method | Links | |
|---|---|---|
| Consensus-WUCSModel=Qwen2.5-32B-Coder2025.02 | 83.7 | |
| CodeRSAModel=Qwen2.5-32B-Coder2025.02 | 83.5 | |
| CoderModel=Qwen2.5-32B-Coder2025.02 | 77.6 | |
| CodeRSAModel=Qwen2.5-7B2025.02 | 76.5 | |
| RandomModel=Qwen2.5-32B-Coder2025.02 | 74.3 | |
| CoderModel=Qwen2.5-7B2025.02 | 73.8 | |
| CoderReviewerModel=Qwen2.5-7B2025.02 | 73.8 | |
| Consensus-WUCSModel=Qwen2.5-7B2025.02 | 72.4 | |
| CodeRSAModel=Llama-3-70B2025.02 | 70.8 | |
| Consensus-WUCSModel=Llama-3-70B2025.02 | 70.4 | |
| CoderReviewerModel=Qwen2.5-32B-Coder2025.02 | 69.6 | |
| RandomModel=Qwen2.5-7B2025.02 | 68.5 | |
| CoderReviewerModel=Llama-3-70B2025.02 | 65 | |
| CoderModel=Llama-3-70B2025.02 | 63.9 | |
| RandomModel=Llama-3-70B2025.02 | 61.1 | |
| CodeRSAModel=Llama-3-8B2025.02 | 44.6 | |
| Consensus-WUCSModel=Llama-3-8B2025.02 | 41.2 | |
| CoderReviewerModel=Llama-3-8B2025.02 | 40.5 | |
| CoderModel=Llama-3-8B2025.02 | 40.3 | |
| RandomModel=Llama-3-8B2025.02 | 32.6 |