Code Correctness Evaluation on VP
66F1 ScoreCoCoA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoCoAEval Model=Llama-3.1-8B, lambda=Best2024.10 | 66 | 50.4 | |
| CoCoAEval Model=GPT-4o-mini, lambda=Best2024.10 | 65.5 | 49.3 | |
| CA-SPEval Model=Llama-3.1-8B2024.10 | 63.4 | 52.6 | |
| ICE-ScoreEval Model=Llama-3.1-8B2024.10 | 61.2 | 50.3 | |
| BCEval Model=Llama-3.1-8B2024.10 | 60.9 | 54.8 | |
| CoCoAEval Model=Llama-3.1-8B, lambda=Worst2024.10 | 59.7 | 47.2 | |
| CoCoAEval Model=Qwen-2.5-7B, lambda=Best2024.10 | 47.5 | 44 | |
| VanillaEval Model=Llama-3.1-8B2024.10 | 39.4 | 53.5 | |
| CodeJudgeEval Model=Llama-3.1-8B2024.10 | 35.3 | 49.9 | |
| CoCoAEval Model=GPT-4o-mini, lambda=Worst2024.10 | 34.3 | 44.3 | |
| 2PREval Model=Llama-3.1-8B2024.10 | 32.7 | 51.6 | |
| CodeJudgeEval Model=Qwen-2.5-7B2024.10 | 18.5 | 39.8 | |
| CodeJudgeEval Model=GPT-4o-mini2024.10 | 18.1 | 46.2 | |
| CoCoAEval Model=Qwen-2.5-7B, lambda=Worst2024.10 | 17.6 | 48.1 | |
| ICE-ScoreEval Model=Qwen-2.5-7B2024.10 | 16.7 | 44.8 | |
| CA-SPEval Model=Qwen-2.5-7B2024.10 | 16.1 | 43.9 | |
| 2PREval Model=Qwen-2.5-7B2024.10 | 13.5 | 44.2 | |
| BCEval Model=GPT-4o-mini2024.10 | 12.4 | 46.8 | |
| CA-SPEval Model=GPT-4o-mini2024.10 | 12 | 46.8 | |
| ICE-ScoreEval Model=GPT-4o-mini2024.10 | 9.7 | 43.2 | |
| VanillaEval Model=Qwen-2.5-7B2024.10 | 4.7 | 46.7 | |
| BCEval Model=Qwen-2.5-7B2024.10 | 2.2 | 48 | |
| 2PREval Model=GPT-4o-mini2024.10 | 1.7 | 48.7 | |
| VanillaEval Model=GPT-4o-mini2024.10 | 1.1 | 49.7 |