List your company
ResearchFormerly WizWand

Reasoning on StrategyQA

83.5AccuracyCoT-SC
-3.3419.20541.7564.295Nov 15, 2023Apr 22, 2024Sep 28, 2024Mar 7, 2025Aug 13, 2025Jan 19, 2026Jun 28, 2026
Updated 4mo ago

Evaluation results

MethodLinks
CoT-SCModel=GPT-42023.1183.5
StrategyLLM-SCModel=GPT-42023.1183.5
JT-Safe-V2-35BParameters=35B2026.0583.19
StrategyLLMModel=GPT-42023.1181.5
CoTModel=GPT-42023.1180.5
StrategyLLM-SCModel=Claude-3-Sonnet2023.1177
SolutionLLMModel=GPT-42023.1175.5
CoT-SCModel=Claude-3-Sonnet2023.1175
StrategyLLMModel=Claude-3-Sonnet2023.1175
SolutionLLMModel=Claude-3-Sonnet2023.1173.5
RM-RegenBase Model=Llama 3.1-8B2026.0373
RM-RegenBase Model=GPT-3.52026.0369.75
CoTModel=Claude-3-Sonnet2023.1169
ST CoTBase Model=Llama 3.1-8B, Iterations=22026.0368.75
ST CoTBase Model=Llama 3.1-8B, Iterations=32026.0368
Zero-shot-CPPrompting Strategy=Zero-shot-CP, Base Model=GPT-3.5-Turbo, Number of runs=52024.0367.39
ST CoTBase Model=Llama 3.1-8B, Iterations=42026.0366.5
SPEARModel=Qwen2.5-1.5B2026.0566.08
ST CoTBase Model=Llama 3.1-8B, Iterations=52026.0366
SPEARModel=Llama3.2-3B2026.0565.84
Gen-SSD2026.0465.68
TILRBackbone=GPT-2, Average generation tokens per inference=22026.0665.5
Zero-shotPrompting Strategy=Zero-shot, Base Model=GPT-3.5-Turbo, Number of runs=52024.0365.02
MoRSD2026.0464.93
ProCoBase Model=Llama 3.1-8B, Iterations=42026.0364.75
MCC-KD2026.0464.37
Standard KD2026.0463.23
RefinementBackbone=GPT-2, Average generation tokens per inference=22026.0662.4
SOTA with Equivalent ParametersModel comparison=Equivalent Parameters2026.0562.36
ST CoTBase Model=GPT-3.5, Iterations=22026.0362
ST CoTBase Model=GPT-3.5, Iterations=32026.0362
ProCoBase Model=Llama 3.1-8B, Iterations=32026.0362
ProCoBase Model=Llama 3.1-8B, Iterations=52026.0361.25
Zero-shot-CoTPrompting Strategy=Zero-shot-CoT, Base Model=GPT-3.5-Turbo, Number of runs=52024.0360.74
ST CoTBase Model=GPT-3.5, Iterations=42026.0360.25
ProCoBase Model=GPT-3.5, Iterations=32026.0360
CoconutBackbone=GPT-2, Average generation tokens per inference=22026.0660
ProCoBase Model=Llama 3.1-8B, Iterations=22026.0359.75
ProCoBase Model=GPT-3.5, Iterations=22026.0356.75
AdaAnchorBackbone=GPT-2, Average generation tokens per inference=22026.0656.1
ProCoBase Model=GPT-3.5, Iterations=42026.0355.75
CoTBackbone=GPT-2, Average generation tokens per inference=222026.0655.2
Self-RefineBase Model=Llama 3.1-8B, Iterations=42026.0355
RLTF-SDModel=Qwen2.5-1.5B2026.0554.29
Self-RefineBase Model=Llama 3.1-8B, Iterations=22026.0354.25
Self-RefineBase Model=Llama 3.1-8B, Iterations=32026.0353.75
No-CoTBackbone=GPT-2, Average generation tokens per inference=12026.0652.3
RLTF-SDModel=Llama3.2-3B2026.0551.58
GRPOModel=Qwen2.5-1.5B2026.0551.38
Self-RefineBase Model=Llama 3.1-8B, Iterations=52026.0351.25
Feedback SFTModel=Qwen2.5-1.5B2026.0550.51
Feedback SFTModel=Llama3.2-3B2026.0549.49
GRPOModel=Llama3.2-3B2026.0549.34
Self-RefineBase Model=GPT-3.5, Iterations=42026.0344.75
Self-RefineBase Model=GPT-3.5, Iterations=22026.0344
Self-RefineBase Model=GPT-3.5, Iterations=32026.0341.75
OPSDModel=Llama3.2-3B2026.050
OPSDModel=Qwen2.5-1.5B2026.050