List your company
ResearchFormerly WizWand

Logical Reasoning on Stepgame k=10

88.1AccuracyLLM-ASP
6.9828.0449.170.16Dec 23, 2024
Updated 7mo ago

Evaluation results

MethodLinks
LLM-ASPLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.1288.1
PoT-SymbolicLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.1287.8
PoT-SymbolicLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.1287.8
PoT-SymbolicLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.1287.8
PoT-SymbolicLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.1286.3
PoT-SymbolicLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.1285.6
PoT-SymbolicLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.1284.5
LLM-ASPLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.1283.7
PoT-LLMLLM=GPT-5-Medium, Framework=Prompting-based2024.1283.7
LLM-ASPLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.1283.2
CoT-SCLLM=GPT-5-Medium, Framework=Prompting-based2024.1283
Few-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.1282.2
LLM-ASPLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.1282.1
CoTLLM=GPT-5-Medium, Framework=Prompting-based2024.1282
Zero-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.1281.4
LLM-ASPLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.1281.1
PoT-SymbolicLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.1280.3
LLM-ASPLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.1272.6
LLM-ASPLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.1271.6
PoT-LLMLLM=GPT-5-Minimal, Framework=Prompting-based2024.1268.9
CoT-SCLLM=GPT-5-Minimal, Framework=Prompting-based2024.1267.4
PoT-SymbolicLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.1266
LLM-ASPLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.1263.5
CoTLLM=GPT-5-Minimal, Framework=Prompting-based2024.1260.5
PoT-LLMLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1257.2
PoT-LLMLLM=GPT-o1-mini, Framework=Prompting-based2024.1249.3
PoT-LLMLLM=GPT-4o, Category=Prompting-based2024.1248.7
CoT-SCLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1246
CoTLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1244.5
Zero-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.1242
PoT-LLMLLM=Llama3-70B, Category=Prompting-based2024.1241.8
Few-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1240.9
CoT-SCLLM=GPT-4o, Category=Prompting-based2024.1240.4
PoT-LLMLLM=GPT-4-turbo, Category=Prompting-based2024.1240.1
CoTLLM=GPT-4o, Category=Prompting-based2024.1240
CoT-SCLLM=GPT-o1-mini, Framework=Prompting-based2024.1239.8
Few-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.1238.3
Zero-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1237.8
Zero-shotLLM=GPT-4o, Category=Prompting-based2024.1237.7
CoTLLM=GPT-o1-mini, Framework=Prompting-based2024.1237.5
CoT-SCLLM=GPT-4-turbo, Category=Prompting-based2024.1234.4
CoTLLM=GPT-4-turbo, Category=Prompting-based2024.1234.2
CoT-SCLLM=Llama3-70B, Category=Prompting-based2024.1232.9
Zero-shotLLM=GPT-4-turbo, Category=Prompting-based2024.1232.8
CoTLLM=Llama3-70B, Category=Prompting-based2024.1231.6
Few-shotLLM=GPT-4-turbo, Category=Prompting-based2024.1229.8
Few-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.1229.6
PoT-LLMLLM=GPT-3.5-turbo, Category=Prompting-based2024.1228.8
Zero-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.1227.5
Few-shotLLM=GPT-4o, Category=Prompting-based2024.1226.6
Few-shotLLM=Llama3-70B, Category=Prompting-based2024.1225.7
CoT-SCLLM=GPT-3.5-turbo, Category=Prompting-based2024.1221.7
CoTLLM=GPT-3.5-turbo, Category=Prompting-based2024.1219.3
Zero-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.1217
Few-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.1216.8
Zero-shotLLM=Llama3-70B, Category=Prompting-based2024.1210.1