Commonsense Reasoning on CommonsenseQA (pass@1 accuracy)
86.6Accuracy (pass@1)DENOISE
Evaluation Results
| Method | Links | |
|---|---|---|
| DENOISEModel=DeepSeek-V3, K=52026.04 | 86.6 | |
| FACT-EModel=DeepSeek-V3, K=52026.04 | 86.2 | |
| BASEModel=DeepSeek-V3, K=52026.04 | 86 | |
| REFLECTModel=DeepSeek-V3, K=52026.04 | 86 | |
| EVOSELECTIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 85.59 | |
| EVOSELECTIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 85.59 | |
| DiversityIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 85.42 | |
| SwiRBackbone=Qwen3-8B2025.10 | 85.34 | |
| AttributionIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 85.34 | |
| Attr-DivIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 85.26 | |
| Llama-3.3-70B-InstructShots=72026.04 | 85.26 | |
| AttributionIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 85.18 | |
| AttributionIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 85.09 | |
| RandomIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 85.09 | |
| DiversityIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 85.09 | |
| Attr-DivIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 84.93 | |
| EVOSELECTIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 84.93 | |
| POLISHModel=DeepSeek-V3, K=52026.04 | 84.8 | |
| AllIteration=1, Model Scale=14b2026.04 | 84.77 | |
| RandomIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 84.68 | |
| AttributionIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 84.68 | |
| RandomIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 84.6 | |
| EVOSELECTIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 84.52 | |
| RandomIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 84.52 | |
| Attr-DivIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 84.44 | |
| TSDSIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 84.44 | |
| DiversityIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 84.36 | |
| TSDSIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 84.36 | |
| TSDSIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 84.28 | |
| Attr-DivIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 84.19 | |
| AllIteration=2, Model Scale=14b2026.04 | 84.11 | |
| FACT-EModel=4o-mini, K=52026.04 | 84 | |
| CoTBackbone=Qwen3-8B2025.10 | 83.95 | |
| CoT (Greedy)Backbone=Qwen3-8B, decoding=Greedy2025.10 | 83.95 | |
| CONSIST.Model=4o-mini, K=52026.04 | 83.8 | |
| Soft ThinkingBackbone=Qwen3-8B2025.10 | 83.7 | |
| TSDSIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 83.7 | |
| DiversityIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 83.54 | |
| BaseModel Scale=14b2026.04 | 83.13 | |
| p*LLM Family=Qwen 3 (1.7B and 14B)2026.04 | 83.1 | |
| BASEModel=4o-mini, K=52026.04 | 83 | |
| DENOISEModel=4o-mini, K=52026.04 | 82.8 | |
| Qwen3.5-9BShots=72026.04 | 82.64 | |
| Qwen3-8BShots=72026.04 | 82.56 | |
| FACT-EModel=Qwen3, K=52026.04 | 82.4 | |
| Qwen3-14BShots=72026.04 | 82.31 | |
| CONSIST.Model=DeepSeek-V3, K=52026.04 | 82 | |
| POLISHModel=4o-mini, K=52026.04 | 81.8 | |
| REFLECTModel=4o-mini, K=52026.04 | 81.8 | |
| CONSIST.Model=Qwen3, K=52026.04 | 81.8 | |
| BASEModel=Qwen3, K=52026.04 | 81.4 | |
| Dual KADLLM Family=Qwen 3 (1.7B and 14B), lambda=0.32026.04 | 81.3 | |
| DENOISEModel=Qwen3, K=52026.04 | 81 | |
| POLISHModel=Qwen3, K=52026.04 | 81 | |
| REFLECTModel=Qwen3, K=52026.04 | 80.8 | |
| Dual KADLLM Family=Qwen 3 (1.7B and 14B), lambda=0.42026.04 | 80.7 | |
| XekRung-8BShots=72026.04 | 80.59 | |
| SecGPT-14BShots=72026.04 | 79.52 | |
| NudgingLLM Family=Qwen 3 (1.7B and 14B), lambda=0.42026.04 | 79.3 | |
| confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=0.22026.04 | 79.3 | |
| confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=0.12026.04 | 79.2 | |
| NudgingLLM Family=Qwen 3 (1.7B and 14B), lambda=0.32026.04 | 78.5 | |
| Imp. rewardLLM Family=Qwen 3 (1.7B and 14B)2026.04 | 78.1 | |
| p*LLM Family=OLMo 2 (1B and 13B)2026.04 | 76.9 | |
| pLLM Family=Qwen 3 (1.7B and 14B)2026.04 | 76.9 | |
| confidence betLLM Family=Qwen 3 (1.7B and 14B), lambda=02026.04 | 74.7 | |
| Zero-shot-CPPrompting Strategy=Zero-shot-CP, Base Model=GPT-3.5-Turbo, Number of runs=52024.03 | 73.81 | |
| Zero-shotPrompting Strategy=Zero-shot, Base Model=GPT-3.5-Turbo, Number of runs=52024.03 | 71.33 | |
| Zero-shot-CoTPrompting Strategy=Zero-shot-CoT, Base Model=GPT-3.5-Turbo, Number of runs=52024.03 | 68.96 | |
| Full PrecisionModel=LLaMA-2-7B, Wbit/Abit=16/162026.04 | 68.9 | |
| q*LLM Family=Qwen 3 (1.7B and 14B)2026.04 | 68.7 | |
| FACT-EModel=ChatGpt, K=52026.04 | 63.4 | |
| confidence betLLM Family=OLMo 2 (1B and 13B), lambda=0.12026.04 | 58.7 | |
| Imp. rewardLLM Family=OLMo 2 (1B and 13B)2026.04 | 55.8 | |
| CONSIST.Model=ChatGpt, K=52026.04 | 55.6 | |
| Dual KADLLM Family=OLMo 2 (1B and 13B), lambda=0.42026.04 | 55.6 | |
| confidence betLLM Family=OLMo 2 (1B and 13B), lambda=0.22026.04 | 54.9 | |
| confidence betLLM Family=OLMo 2 (1B and 13B), lambda=02026.04 | 53.4 | |
| REFLECTModel=ChatGpt, K=52026.04 | 53.2 | |
| Dual KADLLM Family=OLMo 2 (1B and 13B), lambda=0.32026.04 | 52.3 | |
| POLISHModel=ChatGpt, K=52026.04 | 52.22 | |
| DENOISEModel=ChatGpt, K=52026.04 | 51.6 | |
| BASEModel=ChatGpt, K=52026.04 | 49.6 | |
| q*LLM Family=OLMo 2 (1B and 13B)2026.04 | 48.4 | |
| BitnetModel=Bitnet-b1.58-1.3B, Wbit/Abit=1.5/82026.04 | 47.2 | |
| Nautile-370MTraining tokens=∼0.8T, Evaluation Protocol=0-shot2026.04 | 46.8 | |
| Qwen2.5 0.5BTraining tokens=18T, Evaluation Protocol=0-shot2026.04 | 46.5 | |
| NudgingLLM Family=OLMo 2 (1B and 13B), lambda=0.42026.04 | 46.3 | |
| BWTAModel=Bitnet-b1.58-1.3B, Wbit/Abit=1.3/62026.04 | 44.8 | |
| BitnetModel=Bitnet-b1.58-0.7B, Wbit/Abit=1.5/82026.04 | 44.4 | |
| LFM2.5 350MTraining tokens=28T, Evaluation Protocol=0-shot2026.04 | 44.3 | |
| BWTAModel=Bitnet-b1.58-0.7B, Wbit/Abit=1.3/62026.04 | 43.4 | |
| BWNModel=Bitnet-b1.58-1.3B, Wbit/Abit=1.3/62026.04 | 42.5 | |
| BiLLMModel=LLaMA-2-7B, Wbit/Abit=1.1/162026.04 | 42.3 | |
| Gen-SSD2026.04 | 42.07 | |
| Standard KD2026.04 | 41.66 | |
| MCC-KD2026.04 | 40.67 | |
| Llama-Primus-Reasoning-8BShots=72026.04 | 40.46 | |
| MoRSD2026.04 | 40.43 | |
| BWNModel=Bitnet-b1.58-0.7B, Wbit/Abit=1.3/62026.04 | 40.4 |