Symbolic Reasoning on COLOR (Colored Object)
99.4AccuracySATLM
Evaluation Results
| Method | Links | |
|---|---|---|
| SATLMLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 99.4 | |
| PROGLMLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 98 | |
| SATLMLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 97.7 | |
| PROGLMLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 95.1 | |
| COTLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 90.6 | |
| COTLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 86.3 | |
| STANDARDLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 75.7 |