Math Programming on GSM8K Python
78.5Pass@100Minerva
Evaluation results
| Method | Links | |
|---|---|---|
| MinervaLearning mode=Few-shot learning, Model size=540B, Generation strategy=majority voting2023.05 | 78.5 | |
| CodeT5+Learning mode=Finetuning, Model size=770M2023.05 | 73.8 | |
| code-davinciLearning mode=Few-shot learning2023.05 | 71 | |
| CodeT5+Learning mode=Finetuning, Model size=220M2023.05 | 70.5 | |
| LLaMALearning mode=Few-shot learning, Model size=65B, Generation strategy=majority voting2023.05 | 69.7 | |
| MinervaLearning mode=Few-shot learning, Model size=62B, Generation strategy=majority voting2023.05 | 68.5 | |
| CodeT5Learning mode=Finetuning, Model size=220M2023.05 | 58.4 | |
| LLaMALearning mode=Few-shot learning, Model size=33B, Generation strategy=majority voting2023.05 | 53.1 | |
| CodeGen-monoLearning mode=Finetuning, Model size=2B2023.05 | 47.8 | |
| GPT-NeoLearning mode=Finetuning, Model size=2.7B, Generation strategy=self-sampling optimization2023.05 | 41.4 | |
| CodeGen-monoLearning mode=Finetuning, Model size=350M2023.05 | 38.7 | |
| LLaMALearning mode=Few-shot learning, Model size=13B, Generation strategy=majority voting2023.05 | 29.3 | |
| MinervaLearning mode=Few-shot learning, Model size=8B, Generation strategy=majority voting2023.05 | 28.4 |