Commonsense Reasoning on Last Letters (test)
92.98AccuracyOurs (Natural Program (NP), No Verification)
Evaluation Results
| Method | Links | |
|---|---|---|
| Ours (Natural Program (NP), No Verification)LLM=GPT-3.5-turbo, Prompting Format=Natural Program (NP), Verification Method=None, Voting Strategy=Majority Voting, k=102023.06 | 92.98 | |
| Ours (NP + Deductive Verification + UPV)LLM=GPT-3.5-turbo, Prompting Format=Natural Program (NP), Verification Method=Deductive Verification, Voting Strategy=UPV, k=102023.06 | 92.6 | |
| CoT + VotingLLM=GPT-3.5-turbo, Prompting Format=Chain-of-Thought (CoT), Voting Strategy=Majority Voting, k=102023.06 | 81.6 |