Code Generation on DS-1000 Python
39.7Pass@1INTERVENOR
Evaluation Results
| Method | Links | |
|---|---|---|
| INTERVENOR2023.11 | 39.7 | |
| GPT-3.52023.11 | 29.7 | |
| WizardCoder2023.11 | 28.4 | |
| CodeLLama2023.11 | 28 | |
| StarCoder2023.11 | 26 | |
| CodeGen2023.11 | 8.4 | |
| InCoder2023.11 | 7.4 |
Loading the SOTA2 catalog…