Code Refinement on HE-PY
27.08PRCoCoA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoCoAlambda=2, Evaluator Model=Llama-3.1-8B2024.10 | 27.08 | 63 | |
| CoCoAlambda=5, Evaluator Model=Llama-3.1-8B2024.10 | 22.91 | 61 | |
| CoCoAlambda=5, Evaluator Model=Qwen-2.5-7B2024.10 | 12.5 | 83 | |
| CoCoAlambda=5, Evaluator Model=GPT-4o-mini2024.10 | 11.27 | 86 | |
| CoCoAlambda=2, Evaluator Model=Qwen-2.5-7B2024.10 | 11.2 | 84 | |
| CoCoAlambda=2, Evaluator Model=GPT-4o-mini2024.10 | 10.15 | 90 | |
| CA-SPEvaluator Model=Llama-3.1-8B2024.10 | 3.51 | 81 | |
| BCEvaluator Model=Llama-3.1-8B2024.10 | 2.19 | 75 | |
| CodeJudgeEvaluator Model=Qwen-2.5-7B2024.10 | 2.01 | 81 | |
| CA-SPEvaluator Model=Qwen-2.5-7B2024.10 | 1.81 | 82 | |
| BCEvaluator Model=Qwen-2.5-7B2024.10 | 1.54 | 80 | |
| BCEvaluator Model=GPT-4o-mini2024.10 | 1.35 | 82 | |
| CA-SPEvaluator Model=GPT-4o-mini2024.10 | 1.26 | 81 | |
| CodeJudgeEvaluator Model=GPT-4o-mini2024.10 | 0.98 | 83 | |
| CodeJudgeEvaluator Model=Llama-3.1-8B2024.10 | 0.83 | 85 |