Language-to-code generation on GSM8k (dev)
84.1Execution AccuracyLEVER
Evaluation results
| Method | Links | |
|---|---|---|
| LEVERBase model=codex-davinci-002, Verification strategy=LEVER, Verifier Base model=RoBERTa-large2023.02 | 84.1 | |
| EP + MLBase model=codex-davinci-002, Decoding/Verification strategy=EP + ML2023.02 | 72.1 | |
| GreedyBase model=codex-davinci-002, Decoding strategy=Greedy2023.02 | 68.1 | |
| Neo-2.7B + SSFinetuning status=With Finetuning, Base model=GPT-Neo-2.7B, Decoding strategy=SS2023.02 | 20.7 |