Code Generation on CSN Python (test)
35.1ROUGE-1 F1GAP-Gen CodeT5
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GAP-Gen CodeT5pipeline=GAP-Gen, backbone=CodeT5, mode=fine-tuning2022.01 | 35.1 | 14.9 | 30.6 | 22.3 | 24.1 | |
| CodeT5mode=fine-tuning2022.01 | 34.6 | 14.6 | 30.2 | 21.6 | 23.4 | |
| GAP-Gen T5pipeline=GAP-Gen, backbone=T5, mode=fine-tuning2022.01 | 31.1 | 12.1 | 27.9 | 21.2 | 22.1 | |
| T5mode=fine-tuning2022.01 | 30.4 | 11.7 | 27.4 | 20.7 | 21.7 | |
| PyMT5mode=fine-tuning2022.01 | 28.4 | 13.5 | 24.8 | 8.6 | — | |
| GPT2mode=fine-tuning2022.01 | 20.9 | 7.6 | 21.9 | 2.8 | — |