Mathematical Reasoning on ALGEBRA
90.9AccuracySATLM
Evaluation results
| Method | Links | |
|---|---|---|
| SATLMLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 90.9 | |
| SATLMLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 77.5 | |
| COTLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 64.9 | |
| PROGLMLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 57.7 | |
| COTLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 53.6 | |
| PROGLMLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 52.3 | |
| STANDARDLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 45.9 |