Code Generation on CodeContest Hard
73.8PRMC-Tree-Of-Agents
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MC-Tree-Of-AgentsStrategy=Refine2025.02 | 73.8 | 63.46 | |
| MC-Tree-Of-AgentsStrategy=Pruning2025.02 | 73.12 | 59.62 | |
| MC-Tree-Of-Agents2025.02 | 70.49 | 56.41 | |
| SingleBackbone=Claude2025.02 | 68.41 | 55.76 | |
| RethinkMCTS2025.02 | 55.79 | 48.04 | |
| ToT2025.02 | 49.07 | 32.69 | |
| SingleBackbone=GPT4omini2025.02 | 45.72 | 26.92 | |
| RAP2025.02 | 39.99 | 26.92 | |
| LATS2025.02 | 39.1 | 30.77 | |
| Reflexion2025.02 | 34.09 | 21.15 | |
| LDB2025.02 | 33.52 | 16.28 | |
| ZeroShot2025.02 | 28.24 | 19.23 | |
| DareFinetune Method=Dare, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 19.65 | 15.38 | |
| SFT on cluster 2Finetune Method=SFT on cluster 2, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 19.5 | 9.62 | |
| DisenLoRAFinetune Method=DisenLoRA, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 19.43 | 9.62 | |
| TiesFinetune Method=Ties, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 18.71 | 9.62 | |
| SFT on cluster 1Finetune Method=SFT on cluster 1, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 18.12 | 9.62 | |
| SFT on allFinetune Method=SFT on all, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 16.73 | 7.69 | |
| w/o tuningFinetune Method=w/o tuning, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 15.46 | 5.77 | |
| TwinFinetune Method=Twin, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 12.92 | 9.62 | |
| SFT on cluster 0Finetune Method=SFT on cluster 0, Base Model=Meta-llama-3.1-instruct-8b2025.02 | 11.69 | 1.92 |