List your company
ResearchFormerly WizWand

Logical Reasoning on Formal Logic

87.8AccuracyUAB
34.13648.0686275.932Mar 21, 2026Apr 1, 2026Apr 12, 2026Apr 23, 2026May 4, 2026May 15, 2026May 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
UABBackbone Model=GPT-OSS-20B, Budget=N=42026.0587.8
UABBackbone Model=Cohere, Budget=N=42026.0587.3
UniformBackbone Model=GPT-OSS-20B, Budget=N=42026.0586.5
LLM-JudgeBackbone Model=GPT-OSS-20B, Budget=N=42026.0586.5
LLM-JudgeBackbone Model=Cohere, Budget=N=42026.0586.1
UniformBackbone Model=Cohere, Budget=N=42026.0584.3
UABBackbone Model=Gemma3-27B, Budget=N=42026.0584.1
LLM-JudgeBackbone Model=Gemma3-27B, Budget=N=42026.0583.9
UniformBackbone Model=Gemma3-27B, Budget=N=42026.0582.5
N=1Backbone Model=Cohere, Budget=N=12026.0581.4
N=1Backbone Model=Gemma3-27B, Budget=N=12026.0578
OracleModel=Gemma2-9B, Black-Box Friendly=true2026.0477.8
N=1Backbone Model=GPT-OSS-20B, Budget=N=12026.0576.1
OracleModel=Llama3.1-8B, Black-Box Friendly=true2026.0474.1
OracleModel=Qwen2.5-3B, Black-Box Friendly=true2026.0473.5
OracleModel=Llama3.2-3B, Black-Box Friendly=true2026.0471.4
OracleModel=Qwen2.5-7B, Black-Box Friendly=true2026.0467.7
DARModel=Falcon3-7B2026.0360.3
Uncertain PromptBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0360.3
UABModel=Qwen2.5-7B, N=42026.0559.2
DARBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0359
MAD-M2Model=Falcon3-7B2026.0358.7
Society Of MindBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0357.7
Society Of MindModel=Falcon3-7B2026.0357.5
Uncertain PromptModel=Falcon3-7B2026.0357.5
Vote PromptModel=Falcon3-7B2026.0357.5
Vote PromptBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0357.4
LLM-JudgeModel=Qwen2.5-7B, N=42026.0556.9
UniformModel=Qwen2.5-7B, N=42026.0556.5
RandomModel=Qwen2.5-7B, N=42026.0556.2
N=1Model=Qwen2.5-7B, N=12026.0555.7
MAD-M^2Backbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0355.1
LengthModel=Qwen2.5-7B, N=42026.0554
RCSuniModel=Gemma2-9B, Black-Box Friendly=true2026.0452.9
RCSmedoidModel=Gemma2-9B, Black-Box Friendly=true2026.0452.6
Uncertain PromptModel=Llama3.1-8B2026.0352.4
Vote PromptModel=Llama3.1-8B2026.0352.4
GreedyModel=Gemma2-9B, Black-Box Friendly=true2026.0452.4
RCSprobModel=Gemma2-9B, Black-Box Friendly=false2026.0452.4
CEModel=Gemma2-9B, Black-Box Friendly=false2026.0452.1
DARModel=Llama3.1-8B2026.0352
RCSfreqModel=Gemma2-9B, Black-Box Friendly=true2026.0451.8
Vote PromptBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.0351.6
SCModel=Gemma2-9B, Black-Box Friendly=true2026.0451.6
Society Of MindModel=Llama3.1-8B2026.0350
ANLLModel=Gemma2-9B, Black-Box Friendly=false2026.0450
DARBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.0349.7
Majority VoteBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0349.4
RCSmedoidModel=Qwen2.5-7B, Black-Box Friendly=true2026.0449.2
RCSuniModel=Qwen2.5-7B, Black-Box Friendly=true2026.0449.2
RCSprobModel=Qwen2.5-7B, Black-Box Friendly=false2026.0449.2
Uncertain PromptBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.0348.8
Majority VoteModel=Falcon3-7B2026.0348.7
Society Of MindBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.0348.7
CEModel=Qwen2.5-7B, Black-Box Friendly=false2026.0448.7
SCModel=Qwen2.5-7B, Black-Box Friendly=true2026.0448.4
NLLModel=Gemma2-9B, Black-Box Friendly=false2026.0448.1
RCSfreqModel=Qwen2.5-7B, Black-Box Friendly=true2026.0447.9
Single-agentModel=Llama3.1-8B2026.0347.2
UABModel=Qwen2.5-1.5B, N=42026.0547.1
LLM-JudgeModel=Qwen2.5-1.5B, N=42026.0545.8
Single-agentBackbone Model=Llama3.1-8B2026.0345.6
Majority VoteBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.0345.6
Majority VoteModel=Llama3.1-8B2026.0345.5
RCSuniModel=Qwen2.5-3B, Black-Box Friendly=true2026.0444.7
RCSprobModel=Qwen2.5-3B, Black-Box Friendly=false2026.0443.9
RCSmedoidModel=Qwen2.5-3B, Black-Box Friendly=true2026.0443.6
GreedyModel=Qwen2.5-7B, Black-Box Friendly=true2026.0443.6
DARBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0343.4
RCSfreqModel=Qwen2.5-3B, Black-Box Friendly=true2026.0443.4
LengthModel=Qwen2.5-1.5B, N=42026.0543.4
UABModel=Llama3.2-3B, N=42026.0543.4
Uncertain PromptModel=Qwen2.5-3B2026.0342.5
Single-agentBackbone Model=Falcon3-7B2026.0342.5
MAD-M2Model=Qwen2.5-3B2026.0342.3
Majority VoteModel=Qwen2.5-3B2026.0342.3
UniformModel=Qwen2.5-1.5B, N=42026.0542.1
N=1Model=Qwen2.5-1.5B, N=12026.0541.7
CEModel=Qwen2.5-3B, Black-Box Friendly=false2026.0441.3
UniformModel=Llama3.2-3B, N=42026.0541.3
Vote PromptModel=Qwen2.5-3B2026.0340.9
DARModel=Qwen2.5-3B2026.0340.9
Single-agentModel=Falcon3-7B2026.0340.9
RandomModel=Qwen2.5-1.5B, N=42026.0540.9
SCModel=Qwen2.5-3B, Black-Box Friendly=true2026.0440.5
Majority VoteBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0340.1
LengthModel=Llama3.2-3B, N=42026.0539.7
Vote PromptBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0339.4
RCSuniModel=Llama3.1-8B, Black-Box Friendly=true2026.0439.4
Society Of MindModel=Qwen2.5-3B2026.0339.3
MAD-M^2Backbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.0339.2
LLM-JudgeModel=Llama3.2-3B, N=42026.0539.2
RandomModel=Llama3.2-3B, N=42026.0538.8
Society Of MindBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0338.6
RCSmedoidModel=Llama3.1-8B, Black-Box Friendly=true2026.0437.8
Vote PromptModel=Qwen2.5-1.5B2026.0337.7
DARModel=Qwen2.5-1.5B2026.0337.3
N=1Model=Llama3.2-3B, N=12026.0537.2
RCSprobModel=Llama3.1-8B, Black-Box Friendly=false2026.0436.8
DARBackbone Model=Qwen2.5-1.5B, Number of Agents=4, Number of Rounds=22026.0336.2