List your company
ResearchFormerly WizWand

Commonsense Reasoning on CommonsenseQA (pass@1 accuracy)

86.6Accuracy (pass@1)DENOISE
38.55251.02663.575.974Mar 13, 2024Jul 20, 2024Nov 27, 2024Apr 6, 2025Aug 13, 2025Dec 21, 2025Apr 30, 2026
Updated 6mo ago

Evaluation results

MethodLinks
DENOISEModel=DeepSeek-V3, K=52026.0486.6
FACT-EModel=DeepSeek-V3, K=52026.0486.2
BASEModel=DeepSeek-V3, K=52026.0486
REFLECTModel=DeepSeek-V3, K=52026.0486
EVOSELECTIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0485.59
EVOSELECTIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0485.59
DiversityIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0485.42
SwiRBackbone=Qwen3-8B2025.1085.34
AttributionIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0485.34
Attr-DivIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0485.26
Llama-3.3-70B-InstructShots=72026.0485.26
AttributionIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0485.18
AttributionIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0485.09
RandomIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0485.09
DiversityIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0485.09
Attr-DivIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0484.93
EVOSELECTIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0484.93
POLISHModel=DeepSeek-V3, K=52026.0484.8
AllIteration=1, Model Scale=14b2026.0484.77
RandomIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0484.68
AttributionIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0484.68
RandomIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0484.6
EVOSELECTIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0484.52
RandomIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0484.52
Attr-DivIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0484.44
TSDSIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0484.44
DiversityIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0484.36
TSDSIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0484.36
TSDSIteration=2, Selection Ratio=0.5, Model Scale=14b2026.0484.28
Attr-DivIteration=2, Selection Ratio=0.2, Model Scale=14b2026.0484.19
AllIteration=2, Model Scale=14b2026.0484.11
FACT-EModel=4o-mini, K=52026.0484
CoTBackbone=Qwen3-8B2025.1083.95
CoT (Greedy)Backbone=Qwen3-8B, decoding=Greedy2025.1083.95
CONSIST.Model=4o-mini, K=52026.0483.8
Soft ThinkingBackbone=Qwen3-8B2025.1083.7
TSDSIteration=1, Selection Ratio=0.5, Model Scale=14b2026.0483.7
DiversityIteration=1, Selection Ratio=0.2, Model Scale=14b2026.0483.54
BaseModel Scale=14b2026.0483.13
p*LLM Family=Qwen 3 (1.7B and 14B)2026.0483.1
BASEModel=4o-mini, K=52026.0483
DENOISEModel=4o-mini, K=52026.0482.8
Qwen3.5-9BShots=72026.0482.64
Qwen3-8BShots=72026.0482.56
FACT-EModel=Qwen3, K=52026.0482.4
Qwen3-14BShots=72026.0482.31
CONSIST.Model=DeepSeek-V3, K=52026.0482
POLISHModel=4o-mini, K=52026.0481.8
REFLECTModel=4o-mini, K=52026.0481.8
CONSIST.Model=Qwen3, K=52026.0481.8
BASEModel=Qwen3, K=52026.0481.4
Dual KADLLM Family=Qwen 3 (1.7B and 14B), lambda=0.32026.0481.3
DENOISEModel=Qwen3, K=52026.0481
POLISHModel=Qwen3, K=52026.0481
REFLECTModel=Qwen3, K=52026.0480.8
Dual KADLLM Family=Qwen 3 (1.7B and 14B), lambda=0.42026.0480.7
XekRung-8BShots=72026.0480.59
SecGPT-14BShots=72026.0479.52
NudgingLLM Family=Qwen 3 (1.7B and 14B), lambda=0.42026.0479.3
confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=0.22026.0479.3
confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=0.12026.0479.2
NudgingLLM Family=Qwen 3 (1.7B and 14B), lambda=0.32026.0478.5
Imp. rewardLLM Family=Qwen 3 (1.7B and 14B)2026.0478.1
p*LLM Family=OLMo 2 (1B and 13B)2026.0476.9
pLLM Family=Qwen 3 (1.7B and 14B)2026.0476.9
confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=02026.0474.7
Zero-shot-CPPrompting Strategy=Zero-shot-CP, Base Model=GPT-3.5-Turbo, Number of runs=52024.0373.81
Zero-shotPrompting Strategy=Zero-shot, Base Model=GPT-3.5-Turbo, Number of runs=52024.0371.33
Zero-shot-CoTPrompting Strategy=Zero-shot-CoT, Base Model=GPT-3.5-Turbo, Number of runs=52024.0368.96
Full PrecisionModel=LLaMA-2-7B, Wbit/Abit=16/162026.0468.9
q*LLM Family=Qwen 3 (1.7B and 14B)2026.0468.7
FACT-EModel=ChatGpt, K=52026.0463.4
confidence betLLM Family=OLMo 2 (1B and 13B), lambda=0.12026.0458.7
Imp. rewardLLM Family=OLMo 2 (1B and 13B)2026.0455.8
CONSIST.Model=ChatGpt, K=52026.0455.6
Dual KADLLM Family=OLMo 2 (1B and 13B), lambda=0.42026.0455.6
confidence betLLM Family=OLMo 2 (1B and 13B), lambda=0.22026.0454.9
confidence betLLM Family=OLMo 2 (1B and 13B), lambda=02026.0453.4
REFLECTModel=ChatGpt, K=52026.0453.2
Dual KADLLM Family=OLMo 2 (1B and 13B), lambda=0.32026.0452.3
POLISHModel=ChatGpt, K=52026.0452.22
DENOISEModel=ChatGpt, K=52026.0451.6
BASEModel=ChatGpt, K=52026.0449.6
q*LLM Family=OLMo 2 (1B and 13B)2026.0448.4
BitnetModel=Bitnet-b1.58-1.3B, Wbit/Abit=1.5/82026.0447.2
Nautile-370MTraining tokens=∼0.8T, Evaluation Protocol=0-shot2026.0446.8
Qwen2.5 0.5BTraining tokens=18T, Evaluation Protocol=0-shot2026.0446.5
NudgingLLM Family=OLMo 2 (1B and 13B), lambda=0.42026.0446.3
BWTAModel=Bitnet-b1.58-1.3B, Wbit/Abit=1.3/62026.0444.8
BitnetModel=Bitnet-b1.58-0.7B, Wbit/Abit=1.5/82026.0444.4
LFM2.5 350MTraining tokens=28T, Evaluation Protocol=0-shot2026.0444.3
BWTAModel=Bitnet-b1.58-0.7B, Wbit/Abit=1.3/62026.0443.4
BWNModel=Bitnet-b1.58-1.3B, Wbit/Abit=1.3/62026.0442.5
BiLLMModel=LLaMA-2-7B, Wbit/Abit=1.1/162026.0442.3
Gen-SSD2026.0442.07
Standard KD2026.0441.66
MCC-KD2026.0440.67
Llama-Primus-Reasoning-8BShots=72026.0440.46
MoRSD2026.0440.43
BWNModel=Bitnet-b1.58-0.7B, Wbit/Abit=1.3/62026.0440.4