Logical Reasoning on Stepgame k=4
93.8AccuracyLLM-ASP
Evaluation Results
| Method | Links | |
|---|---|---|
| LLM-ASPLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.12 | 93.8 | |
| PoT-SymbolicLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.12 | 93.3 | |
| PoT-SymbolicLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.12 | 93.2 | |
| PoT-SymbolicLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.12 | 92.9 | |
| PoT-SymbolicLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.12 | 92.6 | |
| PoT-SymbolicLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.12 | 92.6 | |
| PoT-SymbolicLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.12 | 92.6 | |
| LLM-ASPLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.12 | 89.8 | |
| CoT-SCLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 89.7 | |
| LLM-ASPLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.12 | 89.6 | |
| PoT-LLMLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 89.5 | |
| LLM-ASPLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.12 | 89.4 | |
| Few-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 89.3 | |
| CoTLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 89 | |
| PoT-SymbolicLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.12 | 88.3 | |
| Zero-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 88.2 | |
| LLM-ASPLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.12 | 87.6 | |
| LLM-ASPLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.12 | 87 | |
| LLM-ASPLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.12 | 84.7 | |
| LLM-ASPLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.12 | 83.7 | |
| CoT-SCLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 82.7 | |
| PoT-LLMLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 76.9 | |
| CoTLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 76.9 | |
| PoT-SymbolicLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.12 | 75.9 | |
| PoT-LLMLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 75.5 | |
| CoT-SCLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 70.6 | |
| CoTLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 70.2 | |
| PoT-LLMLLM=GPT-4o, Category=Prompting-based2024.12 | 68 | |
| PoT-LLMLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 65.8 | |
| Few-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 64.1 | |
| Zero-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 63.5 | |
| Zero-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 63.5 | |
| CoT-SCLLM=GPT-4o, Category=Prompting-based2024.12 | 63.2 | |
| CoTLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 61.8 | |
| PoT-LLMLLM=Llama3-70B, Category=Prompting-based2024.12 | 61.6 | |
| CoTLLM=GPT-4o, Category=Prompting-based2024.12 | 61 | |
| Zero-shotLLM=GPT-4o, Category=Prompting-based2024.12 | 60.1 | |
| PoT-LLMLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 59.8 | |
| CoT-SCLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 59.8 | |
| Few-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 57.9 | |
| Zero-shotLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 52.5 | |
| CoT-SCLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 51.7 | |
| CoTLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 51.3 | |
| Few-shotLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 50.7 | |
| CoTLLM=Llama3-70B, Category=Prompting-based2024.12 | 50.6 | |
| CoT-SCLLM=Llama3-70B, Category=Prompting-based2024.12 | 50.2 | |
| PoT-LLMLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 44.8 | |
| Zero-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 40.9 | |
| Few-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 40.1 | |
| Few-shotLLM=Llama3-70B, Category=Prompting-based2024.12 | 38.2 | |
| Few-shotLLM=GPT-4o, Category=Prompting-based2024.12 | 36.4 | |
| CoT-SCLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 28 | |
| CoTLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 26.7 | |
| Zero-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 22.5 | |
| Few-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 20.8 | |
| Zero-shotLLM=Llama3-70B, Category=Prompting-based2024.12 | 20 |