Symbolic Reasoning on LastLetter (test)
86.96AccuracySGE
Evaluation results
| Method | Links | |
|---|---|---|
| SGEModel=GPT-4, Tool=Code Interpreter2024.05 | 86.96 | |
| Decomp PromptingModel=GPT-4, Tool=Code Interpreter2024.05 | 85.64 | |
| CoT PromptingModel=GPT-4, Tool=Code Interpreter2024.05 | 85.18 | |
| Refine PromptingModel=GPT-4, Tool=Code Interpreter2024.05 | 84.82 | |
| Least-to-MostLLM=GPT-3.5-turbo, Protocol=Zero-shot2024.04 | 83.2 | |
| IO PromptingModel=GPT-4, Tool=Code Interpreter2024.05 | 81.98 | |
| DUPLLM=GPT-3.5-turbo, Protocol=Zero-shot2024.04 | 81.2 | |
| Few-shot Auto-CoTLLM=GPT-3.5-turbo, Protocol=Few-shot2024.04 | 81.2 | |
| Few-shot Manual-CoTLLM=GPT-3.5-turbo, Protocol=Few-shot2024.04 | 74.4 | |
| Zero-shot CoTLLM=GPT-3.5-turbo, Protocol=Zero-shot2024.04 | 60.8 | |
| Zero-shot PS+LLM=GPT-3.5-turbo, Protocol=Zero-shot2024.04 | 60.6 |