List your company
ResearchFormerly WizWand

Commonsense Reasoning on CSQA (Accuracy)

91.2CSQA AccuracyPrevious SoTA
68.3274.2680.286.14Mar 21, 2022Nov 30, 2022Aug 11, 2023Apr 21, 2024Dec 31, 2024Sep 11, 2025May 24, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Previous SoTA2022.0391.2
Weighted MV2026.0487.63
Simple MV2026.0487.47
EMS-SimSorting Strategy=Semantic Similarity2026.0487.47
EMS-RelSorting Strategy=Historical Reliability2026.0487.47
Conf-MV2026.0487.24
BEAMModel=Qwen3-30B-A3B, Beta (β)=0.01, Avg. K=4.232026.0586.4
Qwen3-30B-A3BModel=Qwen3-30B-A3B, Target Top-K (K)=8, Avg. K=8.002026.0586.24
Self-Consistency2026.0485.88
CoT-SCBackbone=gpt-4, n=52026.0485.6
Single Agent2026.0484.95
CoTBackbone=gpt-42026.0484.9
CLoTBackbone=gpt-42026.0484.9
C-CoTBackbone=gpt-42026.0483.9
CoT-SCBackbone=gpt-4o-mini, n=52026.0483.4
ISP-CoTBackbone=gpt-42026.0483.4
LoRABackbone=Qwen2.5-3B2026.0282.8
CLoTBackbone=deepseek-v32026.0482.6
HydraLoRABackbone=Qwen2.5-3B2026.0282.56
MoSLoRABackbone=Qwen2.5-3B2026.0282.47
CLoTBackbone=gpt-4o-mini2026.0482.3
CoT-SCBackbone=deepseek-v3, n=52026.0482.2
ISP-CoTBackbone=gpt-4o-mini2026.0481.9
Self-consistencyModel=GPT-3 (Code-davinci-002)2022.0381.5
CoTBackbone=gpt-4o-mini2026.0481.5
ExGRPOModel=Qwen2.5-7B-Instruct, Model Role=Student Model, Adapter=On-Policy Adapt.2026.0281.45
Qwen1.5-MoEModel=Qwen1.5-MoE-A2.7B, Target Top-K (K)=4, Avg. K=4.002026.0581.33
C-CoTBackbone=deepseek-v32026.0481.2
CoTBackbone=deepseek-v32026.0481.1
C-CoTBackbone=gpt-4o-mini2026.0481
ARBackbone=deepseek-v32026.0481
ExGRPOModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0280.85
BEAMModel=Qwen1.5-MoE-A2.7B, Beta (β)=0.01, Avg. K=1.562026.0580.84
Self-consistencyModel=PaLM-540B2022.0380.7
HypLoRABackbone=Qwen2.5-3B2026.0279.85
Few-ShotPrompting=Few-shot2022.1079.5
TrBackbone=gpt-42026.0479.4
ARBackbone=gpt-42026.0479.1
On-Policy DistillationModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0279.05
CoT-promptingModel=PaLM-540B2022.0379
CoT-promptingModel=GPT-3 (Code-davinci-002)2022.0379
Zero-shot-EIModel=Gemini-1.5-Pro, Model Role=Teacher Model2026.0278.78
ARBackbone=gpt-4o-mini2026.0478.7
RevThinkModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0278.45
DARBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0378.3
Uncertain PromptBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0378.2
MAD-M2Model=Falcon3-7B2026.0378.1
Zero-shotModel=Gemini-1.5-Pro, Model Role=Teacher Model2026.0277.9
Society Of MindModel=Falcon3-7B2026.0377.8
DARModel=Falcon3-7B2026.0377.8
Society Of MindBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0377.8
MAD-M^2Backbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0377.7
ReSTEMBackbone=Qwen2.5-1.5B, Training Iterations=12026.0577.64
Uncertain PromptModel=Falcon3-7B2026.0377.6
Vote PromptModel=Falcon3-7B2026.0377.6
Vote PromptBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0377.6
ZS-CoT (teacher)Model=GPT-4o-mini, #Params=-2026.0577.6
Divide-or-ConquerModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0277.42
ExGRPOModel=Gemma-7B-it, Model Role=Student Model, Adapter=On-Policy Adapt.2026.0277.42
Answer AugModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0277.36
ExGRPOModel=Gemma-7B-it, Model Role=Student Model2026.0276.82
ZS-CoT (teacher)Model=GPT-4o-mini, #Params=-2026.0576.8
Majority VoteModel=Falcon3-7B2026.0376.7
Question AugModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0276.4
Majority VoteBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.0376.4
Rephrase QuestionModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0275.18
Single-agentModel=Falcon3-7B2026.0375
Single-agentBackbone Model=Falcon3-7B2026.0375
RevThinkModel=Gemma-7B-it, Model Role=Student Model2026.0274.53
Auto-CoTPrompting=Automatic Chain-of-Thought2022.1074.4
Distill Step-by-StepModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0274.33
SFT-OracleBackbone=Qwen2.5-1.5B, Training Iterations=12026.0574.2
On-Policy DistillationModel=Gemma-7B-it, Model Role=Student Model2026.0274.12
Manual-CoTPrompting=Manual Chain-of-Thought2022.1073.5
SKDModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0273.12
Distill Step-by-StepModel=Gemma-7B-it, Model Role=Student Model2026.0273.01
Answer AugModel=Gemma-7B-it, Model Role=Student Model2026.0273.01
Zero-ShotPrompting=Zero-shot2022.1072.6
SKDModel=Gemma-7B-it, Model Role=Student Model2026.0272.48
HSIR-SFTBackbone=Qwen2.5-1.5B, Training Iterations=12026.0572.48
MAD-M^2Backbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0372.4
DARModel=Qwen2.5-3B2026.0371.8
Society Of MindModel=Qwen2.5-3B2026.0371.4
Zero-shotModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.0271.33
Majority VoteModel=Qwen2.5-3B2026.0371.3
MAD-M2Model=Qwen2.5-3B2026.0371.2
Majority VoteBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0371.2
Vanilla-KDModel=FlanT5-large, #Params=780M2026.0571.2
Society Of MindBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0371
DARBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.0370.8
Uncertain PromptBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0370.7
Vote PromptModel=Qwen2.5-3B2026.0370.6
DARBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0370.6
Vote PromptBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.0370.4
Rephrase QuestionModel=Gemma-7B-it, Model Role=Student Model2026.0270.22
Uncertain PromptModel=Qwen2.5-3B2026.0370.2
Divide-or-ConquerModel=Gemma-7B-it, Model Role=Student Model2026.0270.15
GateKDModel=FlanT5-base, #Params=250M2026.0569.5
Vanilla-KDModel=T5-large, #Params=780M2026.0569.3
Single-agentModel=Qwen2.5-3B2026.0369.2