List your company
ResearchFormerly WizWand

Logical Reasoning on LogiQA-2

83.8AccuracyGPT-5
38.19650.035561.87573.7145Jul 12, 2024Oct 29, 2024Feb 15, 2025Jun 4, 2025Sep 21, 2025Jan 8, 2026Apr 28, 2026
Updated 6mo ago

Evaluation results

MethodLinks
GPT-5Evaluation Protocol=Closed-Source2026.0183.8
Gemini2.5-ProEvaluation Protocol=Closed-Source2026.0178.9
GPT-4.1Evaluation Protocol=Closed-Source2026.0178.2
SE-GPTModel=GPT-42024.0776.1
GPT-4oEvaluation Protocol=Closed-Source2026.0172.9
Zero-shotModel=GPT-42024.0770.6
Self-EXPModel=GPT-42024.0769.8
Zero-shot-CoTModel=GPT-42024.0768.6
ATLAS (cluster)Evaluation Protocol=In-Distribution2026.0166.8
Self-ICLModel=GPT-42024.0765.1
ATLAS (RL)Evaluation Protocol=Out-of-Distribution2026.0162.7
Self-ICL-CoTModel=GPT-42024.0760.7
Beam Search (BL)Model=Gemma-3-12b-it2026.0460.1
TTPOModel=Gemma-3-12b-it2026.0459.5
TTPOModel=Phi-4-mini-instruct (4B)2026.0459.4
TTPOModel=Phi-4-mini-instruct (4B)2026.0459.4
SteeringModel=Gemma-3-12b-it2026.0459.4
Greedy (BL)Model=Gemma-3-12b-it2026.0459.2
Beam SearchModel=Phi-4-mini-instruct (4B), beam size=52026.0459
Beam Search (BL)Model=Phi-4-mini-instruct (4B)2026.0459
Self-ConsistencyModel=Phi-4-mini-instruct (4B), n=52026.0458.8
BranchingModel=Gemma-3-12b-it2026.0458.5
SteeringModel=Phi-4-mini-instruct (4B)2026.0458.3
GreedyModel=Phi-4-mini-instruct (4B)2026.0457.7
Greedy (BL)Model=Phi-4-mini-instruct (4B)2026.0457.7
BranchingModel=Phi-4-mini-instruct (4B)2026.0457
BranchingModel=Gemma-3-4b-it2026.0456.8
BranchingModel=Gemma-3-4b-it2026.0456.8
TTPOModel=Gemma-3-4b-it2026.0456
SE-GPTModel=GPT-3.52024.0755.6
SteeringModel=Gemma-3-4b-it2026.0455.5
Self-ConsistencyModel=Gemma-3-4b-it, n=52026.0455.3
GreedyModel=Gemma-3-4b-it2026.0455.2
Greedy (BL)Model=Gemma-3-4b-it2026.0455.2
Beam SearchModel=Gemma-3-4b-it, beam size=52026.0454
Beam Search (BL)Model=Gemma-3-4b-it2026.0454
Zero-shot-CoTModel=GPT-3.52024.0752.2
DiversityIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0452.04
Attr-DivIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0451.84
EVOSELECTIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0451.84
Zero-shotModel=GPT-3.52024.0751.6
DiversityIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0451.59
Attr-DivIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0451.53
Self-EXPModel=GPT-3.52024.0751.5
DiversityIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0451.4
RandomIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0451.15
TSDSIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0451.02
EVOSELECTIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0451.02
Modified Self-ICLModel=GPT-3.52024.0751
EVOSELECTIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0450.95
TSDSIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0450.7
Attr-DivIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0450.64
RouterDCEvaluation Protocol=Out-of-Distribution2026.0150.4
AttributionIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0450.38
AttributionIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0450.38
AllIteration=2, Model Scale=14b2026.0450.32
AttributionIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0450.32
RandomIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0450.19
AttributionIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0450.13
EVOSELECTIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0450.13
RandomIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0450
DiversityIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0450
TSDSIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0450
RandomIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0449.87
Beam Search (BL)Model=Phi-4-reasoning-plus (13B)2026.0449.8
AllIteration=1, Model Scale=14b2026.0449.75
TSDSIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0449.43
Random RouterEvaluation Protocol=Training-free2026.0149.2
BaseModel Scale=14b2026.0448.98
Attr-DivIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0448.47
Greedy (BL)Model=Phi-4-reasoning-plus (13B)2026.0447.8
RouterDCEvaluation Protocol=In-Distribution2026.0147.2
BertRouterEvaluation Protocol=In-Distribution2026.0147.1
SteeringModel=Phi-4-reasoning-plus (13B)2026.0447
TTPOModel=Phi-4-reasoning-plus (13B)2026.0445.7
ATLAS (cluster)Evaluation Protocol=Out-of-Distribution2026.0145.6
ZS RouterEvaluation Protocol=Training-free, Prompting Strategy=Zero-shot2026.0145.3
BranchingModel=Phi-4-reasoning-plus (13B)2026.0443.6
AutoP-ICLModel=GPT-3.52024.0742.2
EVOSELECTIteration=2, Selection Ratio=0.2, Model Scale=3b2026.0441.98
AttributionIteration=1, Selection Ratio=0.5, Model Scale=3b2026.0441.79
EVOSELECTIteration=1, Selection Ratio=0.2, Model Scale=3b2026.0441.48
RandomIteration=1, Selection Ratio=0.2, Model Scale=3b2026.0441.22
MLPRouterEvaluation Protocol=In-Distribution2026.0141.2
EVOSELECTIteration=2, Selection Ratio=0.5, Model Scale=3b2026.0441.16
EVOSELECTIteration=1, Selection Ratio=0.5, Model Scale=3b2026.0441.09
RandomIteration=2, Selection Ratio=0.2, Model Scale=3b2026.0441.09
AttributionIteration=2, Selection Ratio=0.5, Model Scale=3b2026.0441.03
RandomIteration=1, Selection Ratio=0.5, Model Scale=3b2026.0440.84
DiversityIteration=2, Selection Ratio=0.5, Model Scale=3b2026.0440.84
FS RouterEvaluation Protocol=Training-free, Prompting Strategy=Few-shot2026.0140.8
DiversityIteration=2, Selection Ratio=0.2, Model Scale=3b2026.0440.65
BertRouterEvaluation Protocol=Out-of-Distribution2026.0140.3
TSDSIteration=1, Selection Ratio=0.2, Model Scale=3b2026.0440.27
TSDSIteration=1, Selection Ratio=0.5, Model Scale=3b2026.0440.2
DiversityIteration=1, Selection Ratio=0.5, Model Scale=3b2026.0440.01
Attr-DivIteration=2, Selection Ratio=0.2, Model Scale=3b2026.0440.01
AllIteration=1, Model Scale=3b2026.0439.95
RandomIteration=2, Selection Ratio=0.5, Model Scale=3b2026.0439.95
TSDSIteration=2, Selection Ratio=0.5, Model Scale=3b2026.0439.95