Code Generation on CodeContest (Pass@5)
35.2Pass@5MapCoder
Evaluation results
| Method | Links | |
|---|---|---|
| MapCoderBase LLM=GPT42024.05 | 35.2 | |
| AlphaCodiumBase LLM=GPT42024.05 | 29 | |
| DirectBase LLM=GPT42024.05 | 18.8 | |
| MapCoderBase LLM=ChatGPT2024.05 | 18.2 | |
| AlphaCodiumBase LLM=ChatGPT2024.05 | 17 | |
| DirectBase LLM=ChatGPT2024.05 | 11.2 |