Mathematical Reasoning on GSM8K-Python (test)
51.3Pass@1PaLM
Evaluation Results
| Method | Links | |
|---|---|---|
| PaLMTraining stage=Pretraining only, Model scale=540B, Shots=4-shot2022.04 | 51.3 | |
| PaLM-CoderTraining stage=Code Finetuning, Model scale=540B, Shots=4-shot2022.04 | 50.9 | |
| Davinci CodexTraining stage=Code Finetuning, Shots=4-shot, Note=Calculation via OpenAI Codex API2022.04 | 32.1 | |
| LaMDATraining stage=Pretraining only, Model scale=137B, Shots=4-shot2022.04 | 7.6 |