Math Programming on MathQA Python
87.4Pass@80CodeT5+
Evaluation results
| Method | Links | |
|---|---|---|
| CodeT5+Learning mode=Finetuning, Model size=770M2023.05 | 87.4 | |
| CodeGen-monoLearning mode=Finetuning, Model size=2B2023.05 | 85.6 | |
| CodeT5+Learning mode=Finetuning, Model size=220M2023.05 | 85.6 | |
| GPT-NeoLearning mode=Finetuning, Model size=125M, Generation strategy=self-sampling optimization2023.05 | 84.7 | |
| CodeGen-monoLearning mode=Finetuning, Model size=350M2023.05 | 83.1 | |
| LaMDALearning mode=Finetuning, Model size=137B2023.05 | 81.2 | |
| CodeT5Learning mode=Finetuning, Model size=220M2023.05 | 71.5 | |
| code-davinciLearning mode=Few-shot learning2023.05 | 42 |