Code-to-text on HumanEvalExplain (test)
64.6Python AccGPT-4
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-4Decoding=Reported scores2023.12 | 64.6 | 57.3 | 51.2 | 58.5 | 38.4 | 42.7 | 52.1 | |
| Deepseek-instruct-6.7BDecoding=Greedy2023.12 | 62.2 | 54.3 | 61 | 39.6 | 55.5 | 33.5 | 51 | |
| Magicoder-S-DSDecoding=Greedy2023.12 | 60.3 | 46.3 | 54.3 | 38.4 | 48.1 | 29.2 | 46.1 | |
| WaveCoder-Ultra-6.7BDecoding=Greedy2023.12 | 56.7 | 50 | 54.3 | 34.8 | 51.2 | 36.6 | 47.3 | |
| Magicoder-DSDecoding=Reported scores2023.12 | 55.5 | 36.6 | 49.4 | 36 | 39.6 | 27.4 | 40.7 | |
| WaveCoder-Pro-6.7BDecoding=Greedy2023.12 | 53 | 43.3 | 54.9 | 34.1 | 42.7 | 20 | 41.3 | |
| WaveCoder-DS-6.7BDecoding=Greedy2023.12 | 48.2 | 47.5 | 49.4 | 32.3 | 48.2 | 22 | 41.3 | |
| WaveCoder-CL-13BDecoding=Reported scores2023.12 | 45.7 | 42 | 48.2 | 32.3 | 38.4 | 20.7 | 37.9 | |
| DeepseekCoder-6.7BDecoding=Reported scores2023.12 | 43.9 | 40.2 | 37.8 | 29.2 | 34.1 | 22.5 | 34.6 | |
| WaveCoder-CL-7BDecoding=Reported scores2023.12 | 41.4 | 31.7 | 39 | 25 | 34.1 | 23.2 | 32.4 | |
| Deepseek-CodeAlpaca-6.7BDecoding=Reported scores2023.12 | 40.8 | 37.2 | 42.1 | 29.9 | 31.7 | 22.5 | 34 | |
| CodeLLaMa-instruct-13BDecoding=Reported scores2023.12 | 40.2 | 26.8 | 37.2 | 22.5 | 28 | 14.6 | 28.2 | |
| WaveCoder-SC-15BDecoding=Reported scores2023.12 | 37.1 | 33.3 | 40.5 | 23.3 | 31.8 | 19.3 | 30.8 | |
| OctoCoderDecoding=Reported scores2023.12 | 35.1 | 24.5 | 27.3 | 21.1 | 24.1 | 14.8 | 24.5 | |
| CodeLLaMa-CodeAlpaca-7BDecoding=Reported scores2023.12 | 34.7 | 24.4 | 37.8 | 23.2 | 28.6 | 19.5 | 28 | |
| CodeLLaMa-instruct-7BDecoding=Reported scores2023.12 | 33.5 | 36 | 31.7 | 21.3 | 25 | 16.4 | 27.3 | |
| WizardCoderDecoding=Reported scores2023.12 | 32.5 | 33 | 27.4 | 26.7 | 28.2 | 16.9 | 27.5 | |
| CodeLLaMa-CodeAlpaca-13BDecoding=Reported scores2023.12 | 32.3 | 28 | 34.1 | 18.9 | 29.9 | 20.7 | 27.3 | |
| StarCoderDecoding=Reported scores2023.12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |