Code Generation on HumanEval+ (mean accuracy)
76.8Mean AccuracyCodeRSA
Evaluation results
| Method | Links | |
|---|---|---|
| CodeRSAModel=Qwen2.5-32B-Coder2025.02 | 76.8 | |
| CoderReviewerModel=Qwen2.5-32B-Coder2025.02 | 75.2 | |
| Consensus-WUCSModel=Qwen2.5-32B-Coder2025.02 | 74.5 | |
| CoderModel=Qwen2.5-32B-Coder2025.02 | 73.5 | |
| RandomModel=Qwen2.5-32B-Coder2025.02 | 71.2 | |
| CodeRSAModel=Llama-3-70B2025.02 | 60.4 | |
| CodeRSAModel=Qwen2.5-7B2025.02 | 59.9 | |
| CoderReviewerModel=Llama-3-70B2025.02 | 55.6 | |
| Consensus-WUCSModel=Qwen2.5-7B2025.02 | 55.4 | |
| CoderModel=Llama-3-70B2025.02 | 52.2 | |
| CoderReviewerModel=Qwen2.5-7B2025.02 | 52 | |
| Consensus-WUCSModel=Llama-3-70B2025.02 | 51.5 | |
| CoderModel=Qwen2.5-7B2025.02 | 49.9 | |
| RandomModel=Qwen2.5-7B2025.02 | 49.5 | |
| CodeRSAModel=Llama-3-8B2025.02 | 47.8 | |
| RandomModel=Llama-3-70B2025.02 | 47.4 | |
| CoderReviewerModel=Llama-3-8B2025.02 | 44.5 | |
| Consensus-WUCSModel=Llama-3-8B2025.02 | 42.9 | |
| CoderModel=Llama-3-8B2025.02 | 40.2 | |
| RandomModel=Llama-3-8B2025.02 | 34.3 |