Logical Reasoning on Stepgame k=10
88.1AccuracyLLM-ASP
Evaluation Results
| Method | Links | |
|---|---|---|
| LLM-ASPLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.12 | 88.1 | |
| PoT-SymbolicLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.12 | 87.8 | |
| PoT-SymbolicLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.12 | 87.8 | |
| PoT-SymbolicLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.12 | 87.8 | |
| PoT-SymbolicLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.12 | 86.3 | |
| PoT-SymbolicLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.12 | 85.6 | |
| PoT-SymbolicLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.12 | 84.5 | |
| LLM-ASPLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.12 | 83.7 | |
| PoT-LLMLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 83.7 | |
| LLM-ASPLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.12 | 83.2 | |
| CoT-SCLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 83 | |
| Few-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 82.2 | |
| LLM-ASPLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.12 | 82.1 | |
| CoTLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 82 | |
| Zero-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.12 | 81.4 | |
| LLM-ASPLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.12 | 81.1 | |
| PoT-SymbolicLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.12 | 80.3 | |
| LLM-ASPLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.12 | 72.6 | |
| LLM-ASPLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.12 | 71.6 | |
| PoT-LLMLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 68.9 | |
| CoT-SCLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 67.4 | |
| PoT-SymbolicLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.12 | 66 | |
| LLM-ASPLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.12 | 63.5 | |
| CoTLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 60.5 | |
| PoT-LLMLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 57.2 | |
| PoT-LLMLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 49.3 | |
| PoT-LLMLLM=GPT-4o, Category=Prompting-based2024.12 | 48.7 | |
| CoT-SCLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 46 | |
| CoTLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 44.5 | |
| Zero-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 42 | |
| PoT-LLMLLM=Llama3-70B, Category=Prompting-based2024.12 | 41.8 | |
| Few-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 40.9 | |
| CoT-SCLLM=GPT-4o, Category=Prompting-based2024.12 | 40.4 | |
| PoT-LLMLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 40.1 | |
| CoTLLM=GPT-4o, Category=Prompting-based2024.12 | 40 | |
| CoT-SCLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 39.8 | |
| Few-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 38.3 | |
| Zero-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.12 | 37.8 | |
| Zero-shotLLM=GPT-4o, Category=Prompting-based2024.12 | 37.7 | |
| CoTLLM=GPT-o1-mini, Framework=Prompting-based2024.12 | 37.5 | |
| CoT-SCLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 34.4 | |
| CoTLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 34.2 | |
| CoT-SCLLM=Llama3-70B, Category=Prompting-based2024.12 | 32.9 | |
| Zero-shotLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 32.8 | |
| CoTLLM=Llama3-70B, Category=Prompting-based2024.12 | 31.6 | |
| Few-shotLLM=GPT-4-turbo, Category=Prompting-based2024.12 | 29.8 | |
| Few-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 29.6 | |
| PoT-LLMLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 28.8 | |
| Zero-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.12 | 27.5 | |
| Few-shotLLM=GPT-4o, Category=Prompting-based2024.12 | 26.6 | |
| Few-shotLLM=Llama3-70B, Category=Prompting-based2024.12 | 25.7 | |
| CoT-SCLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 21.7 | |
| CoTLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 19.3 | |
| Zero-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 17 | |
| Few-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.12 | 16.8 | |
| Zero-shotLLM=Llama3-70B, Category=Prompting-based2024.12 | 10.1 |