Code Generation Critique and Refinement on Critique and Refinement Evaluation Dataset Code
78.3CU ScoreRCO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RCOBackbone=LLaMA-3-8B-Instruct2025.06 | 78.3 | 6.33 | |
| Base modelBackbone=LLaMA-3-70B-Instruct2025.06 | 78.1 | 6.24 | |
| Base modelBackbone=LLaMA-3-8B-Instruct2025.06 | 73.1 | 6.08 | |
| RCOBackbone=LLaMA-2-7B-Chat2025.06 | 72.5 | 5.86 | |
| RCOBackbone=Auto-J-13B2025.06 | 72.3 | 5.63 | |
| DPCOBackbone=LLaMA-3-8B-Instruct2025.06 | 70.5 | 5.32 | |
| DPCOBackbone=Auto-J-13B2025.06 | 70.2 | 5.73 | |
| RCOBackbone=LLaMA-2-13B-Chat2025.06 | 70.1 | 5.69 | |
| RCOBackbone=UltraCM-13B2025.06 | 69.3 | 5.46 | |
| DPCOBackbone=LLaMA-2-13B-Chat2025.06 | 69.2 | 5.7 | |
| DPCOBackbone=UltraCM-13B2025.06 | 67 | 5.33 | |
| Self-refinement2025.06 | 65.8 | 5.06 | |
| Base modelBackbone=Auto-J-13B2025.06 | 63.1 | — | |
| Base modelBackbone=UltraCM-13B2025.06 | 62.4 | 5.25 | |
| DPCOBackbone=LLaMA-2-7B-Chat2025.06 | 62.1 | 5.5 | |
| Base modelBackbone=LLaMA-2-13B-Chat2025.06 | 61.1 | 5.39 | |
| Base modelBackbone=LLaMA-2-70B-Chat2025.06 | 59.7 | 5.59 | |
| Base modelBackbone=LLaMA-2-7B-Chat2025.06 | 59.7 | 5.29 | |
| Aligner2025.06 | 38 | 4.49 | |
| Initial Answer2025.06 | 4.42 | — |