List your company
ResearchFormerly WizWand

Logical Reasoning on Stepgame k=4

93.8AccuracyLLM-ASP
17.04836.97456.976.826Dec 23, 2024
Updated 7mo ago

Evaluation results

MethodLinks
LLM-ASPLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.1293.8
PoT-SymbolicLLM=GPT-5-Medium, Framework=Extraction + Symbolic Reasoing2024.1293.3
PoT-SymbolicLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.1293.2
PoT-SymbolicLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.1292.9
PoT-SymbolicLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.1292.6
PoT-SymbolicLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.1292.6
PoT-SymbolicLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.1292.6
LLM-ASPLLM=Cluade-3.7-sonnet, Framework=Extraction + Symbolic Reasoing2024.1289.8
CoT-SCLLM=GPT-5-Medium, Framework=Prompting-based2024.1289.7
LLM-ASPLLM=Llama3-70B, Category=Extraction + Symbolic Reasoning2024.1289.6
PoT-LLMLLM=GPT-5-Medium, Framework=Prompting-based2024.1289.5
LLM-ASPLLM=GPT-4-turbo, Category=Extraction + Symbolic Reasoning2024.1289.4
Few-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.1289.3
CoTLLM=GPT-5-Medium, Framework=Prompting-based2024.1289
PoT-SymbolicLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.1288.3
Zero-shotLLM=GPT-5-Medium, Framework=Prompting-based2024.1288.2
LLM-ASPLLM=GPT-5-Minimal, Framework=Extraction + Symbolic Reasoing2024.1287.6
LLM-ASPLLM=GPT-o1-mini, Framework=Extraction + Symbolic Reasoing2024.1287
LLM-ASPLLM=GPT-4o, Category=Extraction + Symbolic Reasoning2024.1284.7
LLM-ASPLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.1283.7
CoT-SCLLM=GPT-5-Minimal, Framework=Prompting-based2024.1282.7
PoT-LLMLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1276.9
CoTLLM=GPT-5-Minimal, Framework=Prompting-based2024.1276.9
PoT-SymbolicLLM=GPT-3.5-turbo, Category=Extraction + Symbolic Reasoning2024.1275.9
PoT-LLMLLM=GPT-5-Minimal, Framework=Prompting-based2024.1275.5
CoT-SCLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1270.6
CoTLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1270.2
PoT-LLMLLM=GPT-4o, Category=Prompting-based2024.1268
PoT-LLMLLM=GPT-o1-mini, Framework=Prompting-based2024.1265.8
Few-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1264.1
Zero-shotLLM=Cluade-3.7-sonnet, Framework=Prompting-based2024.1263.5
Zero-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.1263.5
CoT-SCLLM=GPT-4o, Category=Prompting-based2024.1263.2
CoTLLM=GPT-o1-mini, Framework=Prompting-based2024.1261.8
PoT-LLMLLM=Llama3-70B, Category=Prompting-based2024.1261.6
CoTLLM=GPT-4o, Category=Prompting-based2024.1261
Zero-shotLLM=GPT-4o, Category=Prompting-based2024.1260.1
PoT-LLMLLM=GPT-4-turbo, Category=Prompting-based2024.1259.8
CoT-SCLLM=GPT-o1-mini, Framework=Prompting-based2024.1259.8
Few-shotLLM=GPT-o1-mini, Framework=Prompting-based2024.1257.9
Zero-shotLLM=GPT-4-turbo, Category=Prompting-based2024.1252.5
CoT-SCLLM=GPT-4-turbo, Category=Prompting-based2024.1251.7
CoTLLM=GPT-4-turbo, Category=Prompting-based2024.1251.3
Few-shotLLM=GPT-4-turbo, Category=Prompting-based2024.1250.7
CoTLLM=Llama3-70B, Category=Prompting-based2024.1250.6
CoT-SCLLM=Llama3-70B, Category=Prompting-based2024.1250.2
PoT-LLMLLM=GPT-3.5-turbo, Category=Prompting-based2024.1244.8
Zero-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.1240.9
Few-shotLLM=GPT-5-Minimal, Framework=Prompting-based2024.1240.1
Few-shotLLM=Llama3-70B, Category=Prompting-based2024.1238.2
Few-shotLLM=GPT-4o, Category=Prompting-based2024.1236.4
CoT-SCLLM=GPT-3.5-turbo, Category=Prompting-based2024.1228
CoTLLM=GPT-3.5-turbo, Category=Prompting-based2024.1226.7
Zero-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.1222.5
Few-shotLLM=GPT-3.5-turbo, Category=Prompting-based2024.1220.8
Zero-shotLLM=Llama3-70B, Category=Prompting-based2024.1220