Multi-task Language Understanding on MMLU (MMLU Acc)
98.5MMLU AccuracyPass@100
Evaluation results
| Method | Links | |
|---|---|---|
| Pass@100Generator Model=Llama 3.1 8B Instruct, Supervision Setting=Oracle2026.04 | 98.5 | |
| Pass@100Generator Model=Llama 3.3 70B Instruct, Supervision Setting=Oracle2026.04 | 96 | |
| WeaverGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Supervised2026.04 | 94.9 | |
| FUSEGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Unsupervised2026.04 | 94.1 | |
| Naive BayesGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Supervised2026.04 | 93.7 | |
| LogisticGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Supervised2026.04 | 93.1 | |
| Naive EnsembleGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Unsupervised2026.04 | 92.4 | |
| OBV (Oracle Best Verifier)Generator Model=Llama 3.3 70B Instruct, Supervision Setting=Supervised2026.04 | 88.4 | |
| WeaverGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Supervised2026.04 | 85.7 | |
| FUSEGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Unsupervised2026.04 | 84.9 | |
| Majority VoteGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Unsupervised2026.04 | 84.1 | |
| ITIModel=Qwen2.5-32B, Evaluation protocol=5-shot2026.04 | 83.84 | |
| OBV (Oracle Best Verifier)Generator Model=Llama 3.1 8B Instruct, Supervision Setting=Supervised2026.04 | 83.6 | |
| A-LQRModel=Qwen2.5-32B, Evaluation protocol=5-shot2026.04 | 82.82 | |
| CascadeDebateModel Family=Llama-3.22026.04 | 82.67 | |
| Pass@1Generator Model=Llama 3.3 70B Instruct, Supervision Setting=Unsupervised2026.04 | 82.6 | |
| S-PIDModel=Qwen2.5-32B, Evaluation protocol=5-shot2026.04 | 82.46 | |
| OriginalModel=Qwen2.5-32B, Evaluation protocol=5-shot2026.04 | 82.45 | |
| Linear-AcTModel=Qwen2.5-32B, Evaluation protocol=5-shot2026.04 | 82.38 | |
| Naive EnsembleGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Unsupervised2026.04 | 81.9 | |
| S-BoN (b)n=2562025.06 | 81.7 | |
| S-BoN (b)n=642025.06 | 81.4 | |
| S-BoN (b)n=162025.06 | 81 | |
| LogisticGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Supervised2026.04 | 80.4 | |
| ODESteerModel=Qwen-2.5-14B, Evaluation protocol=5-shot2026.04 | 79.72 | |
| OriginalModel=Qwen-2.5-14B, Evaluation protocol=5-shot2026.04 | 79.4 | |
| Linear-AcTModel=Qwen-2.5-14B, Evaluation protocol=5-shot2026.04 | 79.04 | |
| S-BoN (b)n=42025.06 | 78.8 | |
| BF16Model=Qwen3-MoE-30B, Precision=BF16, Evaluation Protocol=Zero-shot2026.05 | 78.76 | |
| Qwen3.5-9BArchitecture Type=Hybrid2026.04 | 78.75 | |
| S-PIDModel=Qwen-2.5-14B, Evaluation protocol=5-shot2026.04 | 78.64 | |
| A-LQRModel=Qwen-2.5-14B, Evaluation protocol=5-shot2026.04 | 78.52 | |
| Mean-AcTModel=Qwen-2.5-14B, Evaluation protocol=5-shot2026.04 | 78.46 | |
| PID-AcTModel=Qwen-2.5-14B, Evaluation protocol=5-shot2026.04 | 78.46 | |
| GSIn=162025.06 | 78.3 | |
| GSIn=2562025.06 | 78.1 | |
| CascadeDebateModel Family=Qwen2.52026.04 | 78 | |
| GSIn=642025.06 | 78 | |
| Naive BayesGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Supervised2026.04 | 77.8 | |
| GSIn=42025.06 | 76.9 | |
| S-BoN (b)n=12025.06 | 76.8 | |
| ITIModel=Qwen-2.5-14B, Evaluation protocol=5-shot2026.04 | 76.14 | |
| FLBackbone=PHI-4-14B, SFT Dataset=UF2026.04 | 75.2 | |
| Qwen2.5Parameters=7B, Complexity Type=Quadratic, Evaluation Framework=vLLM, Evaluation Method=generation-based2025.09 | 75.17 | |
| GEMBackbone=PHI-4-14B, SFT Dataset=UF2026.04 | 74.9 | |
| TOFUBackbone=PHI-4-14B, SFT Dataset=UF2026.04 | 74.9 | |
| CEBackbone=PHI-4-14B, SFT Dataset=UF2026.04 | 74.8 | |
| ANDES (Ours)Data Volume=10k2026.05 | 74.8 | |
| DataflowData Volume=10k2026.05 | 74.6 | |
| Qwen3.5-4BArchitecture Type=Hybrid2026.04 | 74.41 | |
| SARQC-GBS(Identity)Model=Qwen3-MoE-30B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.05 | 74.27 | |
| InfininstructData Volume=10k2026.05 | 74.2 | |
| SpikingBrain-76BParameters=12B/76B, Complexity Type=Hybrid, Evaluation Framework=vLLM, Evaluation Method=generation-based2025.09 | 73.71 | |
| SARQC-GBS(Saliency)Model=Qwen3-MoE-30B, Precision=W3A16, Evaluation Protocol=Zero-shot2026.05 | 73.36 | |
| Standard CascadeModel Family=Qwen2.52026.04 | 73.2 | |
| Base Model(Qwen3-8B-Base)Data Volume=–2026.05 | 73 | |
| Majority VoteGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Unsupervised2026.04 | 72.7 | |
| ATOMBackbone Model=Meta-Llama-3.1-8B-Instruct2026.05 | 72.55 | |
| CEBackbone=PHI-4-14B, SFT Dataset=ALPACA2026.04 | 72 | |
| DPO+FilterBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=12025.10 | 72 | |
| DPO+FilterBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=12025.10 | 72 | |
| DPO+FilterTraining Dataset=UltraFeedback, Pη Type=12025.10 | 72 | |
| DPO+FilterTraining Dataset=HelpSteer3, Pη Type=12025.10 | 72 | |
| GEMBackbone=PHI-4-14B, SFT Dataset=ALPACA2026.04 | 71.9 | |
| TOFUBackbone=PHI-4-14B, SFT Dataset=ALPACA2026.04 | 71.9 | |
| DPO+FilterBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=22025.10 | 71.8 | |
| DPO+FilterTraining Dataset=UltraFeedback, Pη Type=22025.10 | 71.8 | |
| λ-PRBackbone=PHI-4-14B, SFT Dataset=ALPACA2026.04 | 71.7 | |
| GSIn=12025.06 | 71.4 | |
| DPO+FilterBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=22025.10 | 71.4 | |
| DPO+FilterTraining Dataset=HelpSteer3, Pη Type=22025.10 | 71.4 | |
| DPOBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=22025.10 | 71.3 | |
| DPOTraining Dataset=UltraFeedback, Pη Type=22025.10 | 71.3 | |
| InfininstructData Volume=1M2026.05 | 71.3 | |
| DPOBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=12025.10 | 71.1 | |
| BaseBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=-2025.10 | 71.1 | |
| DPOTraining Dataset=UltraFeedback, Pη Type=12025.10 | 71.1 | |
| BaseTraining Dataset=HelpSteer3, Pη Type=-2025.10 | 71.1 | |
| MixtralParameters=13B/47B, Complexity Type=Quadratic, Evaluation Framework=vLLM, Evaluation Method=generation-based2025.09 | 71.03 | |
| FLBackbone=PHI-4-14B, SFT Dataset=ALPACA2026.04 | 71 | |
| DPOBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=22025.10 | 71 | |
| DPOTraining Dataset=HelpSteer3, Pη Type=22025.10 | 71 | |
| FP16#Bits=16-162026.05 | 70.97 | |
| DPOBase=Qwen 7b, Training Dataset=HelpSteer3, Pn Type=12025.10 | 70.8 | |
| DPOTraining Dataset=HelpSteer3, Pη Type=12025.10 | 70.8 | |
| CEBackbone=QWEN-3-8B, SFT Dataset=UF2026.04 | 70.5 | |
| FP16*#Bits=16-16, Evaluation Protocol=original paper result2026.05 | 70.5 | |
| BaseBase=Qwen 7b, Training Dataset=UltraFeedback, Pn Type=-2025.10 | 70.4 | |
| BaseTraining Dataset=UltraFeedback, Pη Type=-2025.10 | 70.4 | |
| FLBackbone=QWEN-3-8B, SFT Dataset=UF2026.04 | 70.3 | |
| TOFUBackbone=QWEN-3-8B, SFT Dataset=UF2026.04 | 70.3 | |
| MSRSBackbone=Qwen2-7B-Instruct2025.08 | 70.2 | |
| GEMBackbone=QWEN-3-8B, SFT Dataset=UF2026.04 | 70.2 | |
| CEBackbone=QWEN-3-8B, SFT Dataset=ALPACA2026.04 | 69.9 | |
| GEMBackbone=QWEN-3-8B, SFT Dataset=ALPACA2026.04 | 69.7 | |
| RSDn=642025.06 | 69.6 | |
| VanillaBackbone=Qwen2-7B-Instruct2025.08 | 69.5 | |
| FLBackbone=QWEN-3-8B, SFT Dataset=ALPACA2026.04 | 69.4 | |
| TOFUBackbone=QWEN-3-8B, SFT Dataset=ALPACA2026.04 | 69.2 | |
| RSDn=42025.06 | 69.1 |