List your company
ResearchFormerly WizWand

Logical Reasoning on Stepgame k=3

89.5AccuracyPoT-LLM
18.57236.98655.473.814Dec 23, 2024
Updated 7mo ago

Evaluation results

MethodLinks
PoT-LLMLLM=GPT-5-Medium, Framework=Prompting-based2024.1289.5
CoTLLM=GPT-5-Medium, Framework=Prompting-based2024.1289.4
CoT-SCLLM=GPT-5-Medium, Framework=Prompting-based2024.1289.3
Few-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.1289.2
LLM-ASPLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.1289.2
PoT-SymbolicLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.1289.2
Zero-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.1289.1
PoT-SymbolicLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.1288.8
PoT-SymbolicLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.1288.2
PoT-SymbolicLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.1288.2
PoT-SymbolicLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.1288
PoT-SymbolicLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.1286.4
LLM-ASPLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.1285.7
LLM-ASPLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.1285.3
LLM-ASPLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.1284.9
LLM-ASPLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.1283.7
PoT-SymbolicLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.1282.8
LLM-ASPLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.1282.4
LLM-ASPLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.1281.1
PoT-LLMLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1281
LLM-ASPLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.1276.4
CoTLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1276
PoT-LLMLLM=GPT-5-Minimal, Framework=Prompting-based2024.1275.6
CoT-SCLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1275.3
CoT-SCLLM=GPT-5-Minimal, Framework=Prompting-based2024.1273.7
PoT-LLMLLM=GPT-4o, Category=Prompting-based2024.1273.4
PoT-LLMLLM=GPT-o1-mini, Framework=Prompting-based2024.1272.9
PoT-SymbolicLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.1272.4
Few-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1271.4
Zero-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1270.6
CoT-SCLLM=GPT-4o, Category=Prompting-based2024.1270
CoTLLM=GPT-4o, Category=Prompting-based2024.1269.4
CoT-SCLLM=GPT-o1-mini, Framework=Prompting-based2024.1269.3
CoTLLM=GPT-5-Minimal, Framework=Prompting-based2024.1269.3
PoT-LLMLLM=Llama3-70B, Category=Prompting-based2024.1269.1
Zero-shotLLM=GPT-4o, Category=Prompting-based2024.1268.6
CoTLLM=GPT-o1-mini, Framework=Prompting-based2024.1268.5
Zero-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.1267.5
Few-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.1267.5
PoT-LLMLLM=GPT-4-turbo, Category=Prompting-based2024.1267.4
CoT-SCLLM=Llama3-70B, Category=Prompting-based2024.1261.3
CoTLLM=Llama3-70B, Category=Prompting-based2024.1260
Zero-shotLLM=GPT-4-turbo, Category=Prompting-based2024.1259
CoTLLM=GPT-4-turbo, Category=Prompting-based2024.1258.3
CoT-SCLLM=GPT-4-turbo, Category=Prompting-based2024.1257.4
Few-shotLLM=GPT-4-turbo, Category=Prompting-based2024.1255.3
PoT-LLMLLM=GPT-3.5-turbo, Category=Prompting-based2024.1250.9
Few-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.1245.1
Zero-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.1244.7
Few-shotLLM=Llama3-70B, Category=Prompting-based2024.1241.2
Few-shotLLM=GPT-4o, Category=Prompting-based2024.1236.6
CoTLLM=GPT-3.5-turbo, Category=Prompting-based2024.1231.1
CoT-SCLLM=GPT-3.5-turbo, Category=Prompting-based2024.1230.7
Zero-shotLLM=Llama3-70B, Category=Prompting-based2024.1224.4
Zero-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.1224
Few-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.1221.3