Logical Reasoning on BBH
100AccuracyUPA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UPAExecutor=GPT-52026.01 | 100 | — | — | |
| IOExecutor=Claude-4.5-Sonnet2026.01 | 100 | — | — | |
| CoTExecutor=Claude-4.5-Sonnet2026.01 | 100 | — | — | |
| UPAExecutor=Claude-4.5-Sonnet2026.01 | 100 | — | — | |
| SPOExecutor=Claude-4.5-Sonnet2026.01 | 99.8 | — | — | |
| UPAExecutor=DeepSeek-V3.22026.01 | 99.7 | — | — | |
| IOExecutor=GPT-52026.01 | 99.5 | — | — | |
| CoTExecutor=GPT-52026.01 | 99.5 | — | — | |
| CoTExecutor=DeepSeek-V3.22026.01 | 99.5 | — | — | |
| IOExecutor=DeepSeek-V3.22026.01 | 99.3 | — | — | |
| SPOExecutor=DeepSeek-V3.22026.01 | 99.3 | — | — | |
| SPOExecutor=GPT-52026.01 | 98.5 | — | — | |
| GoTBackbone=DeepSeek-V32026.03 | 86.2 | — | — | |
| AoTBackbone=DeepSeek-V32026.03 | 86.1 | — | — | |
| AoTBackbone=GPT-4o-mini2026.03 | 86 | — | — | |
| GoTBackbone=GPT-4o-mini2026.03 | 85.9 | — | — | |
| AoTBackbone=Qwen-Turbo2026.03 | 85.4 | — | — | |
| GoTBackbone=Qwen-Turbo2026.03 | 84.9 | — | — | |
| ToTBackbone=DeepSeek-V32026.03 | 84.4 | — | — | |
| ToTBackbone=GPT-4o-mini2026.03 | 84.1 | — | — | |
| ToTBackbone=Qwen-Turbo2026.03 | 83.7 | — | — | |
| CoT-SCBackbone=DeepSeek-V32026.03 | 83.6 | — | — | |
| CoT-SC (n=5)Backbone=GPT-4o-mini2026.03 | 83.4 | — | — | |
| CoT-SC (n=5)Backbone=Qwen-Turbo2026.03 | 83.2 | — | — | |
| TDA-RCBackbone=DeepSeek-V32026.03 | 82.9 | — | — | |
| FoT (n=8)Backbone=DeepSeek-V32026.03 | 82.6 | — | — | |
| TDA-RCBackbone=GPT-4o-mini2026.03 | 82.5 | — | — | |
| FoT (n=8)Backbone=GPT-4o-mini2026.03 | 82.4 | — | — | |
| FoT (n=8)Backbone=Qwen-Turbo2026.03 | 82.3 | — | — | |
| TDA-RCBackbone=Qwen-Turbo2026.03 | 82.2 | — | — | |
| CRDS-RModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 80.44 | — | — | |
| Self-RefineBackbone=DeepSeek-V32026.03 | 80.4 | — | — | |
| CRDS-WModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 80.1 | — | — | |
| Self-RefineBackbone=GPT-4o-mini2026.03 | 80 | — | — | |
| Self-RefineBackbone=Qwen-Turbo2026.03 | 79.8 | — | — | |
| Ling-mini-2.0Model=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=100%, Number of parallel runs=52026.02 | 79.68 | — | — | |
| HY-1.8B-FP16Model Scale=1.8B, Precision=FP16, Instruction-tuned=true2026.02 | 79.08 | — | — | |
| RandomModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 79.01 | — | — | |
| RDS+Model=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 78.72 | — | — | |
| DenseModel=Qwen-3 14B, Sparsity=02025.10 | 78.5 | — | — | |
| CoTBackbone=DeepSeek-V32026.03 | 78.5 | — | — | |
| Mid PPLModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 78.48 | — | — | |
| CoTBackbone=GPT-4o-mini2026.03 | 78.3 | — | — | |
| CoTBackbone=Qwen-Turbo2026.03 | 78.1 | — | — | |
| DenseModel=Qwen-3 8B, Sparsity=02025.10 | 77.42 | — | — | |
| NBDiff-7B-BASEConfiguration=Base2025.12 | 77.3 | — | — | |
| AFlowBackbone=DeepSeek-V32026.03 | 76.4 | — | — | |
| AFlowBackbone=GPT-4o-mini2026.03 | 76 | — | — | |
| AFlowBackbone=Qwen-Turbo2026.03 | 75.7 | — | — | |
| HY-1.8B-2BitModel Scale=1.8B, Precision=2Bit, Instruction-tuned=true2026.02 | 75.54 | — | — | |
| HY-1.8B-INT4Model Scale=1.8B, Precision=INT4, Instruction-tuned=true2026.02 | 74.8 | — | — | |
| FineRMoEBase Model=Qwen2.5-7B, #P/B=26.65, #AP/B=7.942026.03 | 71.22 | — | — | |
| Qwen2.5 7BArchitecture=Autoregressive, Parameters=7B, Number of shots=32026.01 | 70.4 | — | — | |
| C32A2Base Model=Qwen2.5-7B, #P/B=184.42, #AP/B=13.332026.03 | 70.16 | — | — | |
| CTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 69.59 | — | — | |
| PTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 68.3 | — | — | |
| ARMORModel=Qwen-3 14B, Sparsity=2:4+3.89%2025.10 | 68.28 | — | — | |
| BNRMBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=0-shot2026.02 | 67.72 | — | — | |
| DIRBackbone=Llama3.1-8B-Instruct2025.12 | 67.27 | — | — | |
| NVShardBase Model=Qwen2.5-7B, #P/B=47.55, #AP/B=7.632026.03 | 66.67 | — | — | |
| InfoRMBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=0-shot2026.02 | 66.13 | — | — | |
| InfoRMBackbone=Llama3.1-8B-Instruct2025.12 | 66.13 | — | — | |
| SKBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=0-shot2026.02 | 65.69 | — | — | |
| SKBackbone=Llama3.1-8B-Instruct2025.12 | 65.69 | — | — | |
| ALBMBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=0-shot2026.02 | 64.84 | — | — | |
| ALBMBackbone=Llama3.1-8B-Instruct2025.12 | 64.84 | — | — | |
| BaseBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=0-shot2026.02 | 64.52 | — | — | |
| BaseBackbone=Llama3.1-8B-Instruct2025.12 | 64.52 | — | — | |
| DIRBackbone=OpenRLHF-Llama3-8B-SFT2025.12 | 62.99 | — | — | |
| PoEBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=0-shot2026.02 | 62.69 | — | — | |
| PoEBackbone=OpenRLHF-Llama3-8B-SFT2025.12 | 62.69 | — | — | |
| SKBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=0-shot2026.02 | 62.68 | — | — | |
| SKBackbone=OpenRLHF-Llama3-8B-SFT2025.12 | 62.68 | — | — | |
| Qwen2 7BArchitecture=Autoregressive, Parameters=7B, Number of shots=32026.01 | 62.3 | — | — | |
| LPBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=0-shot2026.02 | 62.28 | — | — | |
| LPBackbone=OpenRLHF-Llama3-8B-SFT2025.12 | 62.28 | — | — | |
| LLaMA3 AR 8BArchitecture=Autoregressive, Parameters=8B, Number of shots=32026.01 | 62.1 | — | — | |
| InfoRMBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=0-shot2026.02 | 61.62 | — | — | |
| InfoRMBackbone=OpenRLHF-Llama3-8B-SFT2025.12 | 61.62 | — | — | |
| SparseGPTModel=Qwen-3 14B, Sparsity=2:42025.10 | 61.5 | — | — | |
| BaseBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=0-shot2026.02 | 61.2 | — | — | |
| BaseBackbone=OpenRLHF-Llama3-8B-SFT2025.12 | 61.2 | — | — | |
| LPBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=0-shot2026.02 | 61.1 | — | — | |
| ALBMBase Model=OpenRLHF-Llama3-8B-SFT, Evaluation Protocol=0-shot2026.02 | 61.1 | — | — | |
| LPBackbone=Llama3.1-8B-Instruct2025.12 | 61.1 | — | — | |
| ALBMBackbone=OpenRLHF-Llama3-8B-SFT2025.12 | 61.1 | — | — | |
| Qwen2.5-14BRole=Teacher2026.02 | 60.8 | — | — | |
| PoEBase Model=Llama3.1-8B-Instruct, Evaluation Protocol=0-shot2026.02 | 60.5 | — | — | |
| PoEBackbone=Llama3.1-8B-Instruct2025.12 | 60.5 | — | — | |
| LengthModel=Ling-mini-2.0 (16B), Base Model=Ling-mini-2.0, Truncation Length=2048, Data Selection Ratio=5%, Number of parallel runs=52026.02 | 60.22 | — | — | |
| ARMORModel=Qwen-3 8B, Sparsity=2:4+5.03%2025.10 | 60.13 | — | — | |
| VanillaBackbone=Dream-v0-Instruct-7B, Decode=Vanilla, Cache=None, Block=Naive2026.02 | 59.87 | 17.2 | — | |
| DSB (greedy)Backbone=LLaDA-1.5, Decode=Confidence, Cache=None, Block=DSB (greedy)2026.02 | 58.85 | 69.23 | — | |
| Confidence + NaiveBackbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=None, Block=Naive2026.02 | 58.62 | 96.07 | — | |
| DSB (const.)Backbone=LLaDA-1.5, Decode=Confidence, Cache=None, Block=DSB (const.)2026.02 | 58.53 | 66.16 | — | |
| Dual NaiveBackbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=Dual, Block=Naive2026.02 | 58.47 | 126.67 | — | |
| DSB (greedy)Backbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=None, Block=DSB (greedy)2026.02 | 58.13 | 91.88 | — | |
| HY-0.5B-FP16Model Scale=0.5B, Precision=FP16, Instruction-tuned=true2026.02 | 58.1 | — | — | |
| DSB (const.)Backbone=Dream-v0-Instruct-7B, Decode=Confidence, Cache=None, Block=DSB (const.)2026.02 | 58.09 | 97.31 | — | |
| DSB Cache (greedy)Backbone=LLaDA-1.5, Decode=Confidence, Cache=DSB, Block=DSB (greedy)2026.02 | 58.06 | 120.1 | — |