Multi-task Language Understanding on MMLU-Redux
95.2AccuracyHieraMAS
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| HieraMASMulti=true, Topo=true, Role=true, Node=true, Backbone=GPT-5-Mini2026.02 | 95.2 | — | — | — | — | — | |
| Self-ConsistencyMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 94.4 | — | — | — | — | — | |
| Full-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 94.4 | — | — | — | — | — | |
| CoTMulti=false, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 93.6 | — | — | — | — | — | |
| Self-Consistency+CoTMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 93.6 | — | — | — | — | — | |
| Self-Consistency+CoTMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 92.8 | — | — | — | — | — | |
| LLM-DebateMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 92.8 | — | — | — | — | — | |
| BaseMulti=false, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 92 | — | — | — | — | — | |
| LLM-DebateMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 92 | — | — | — | — | — | |
| GDesignerMulti=true, Topo=true, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 92 | — | — | — | — | — | |
| Random-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 91.2 | — | — | — | — | — | |
| AFlowMulti=true, Topo=true, Role=false, Node=false, Backbone=GPT-5-Mini2026.02 | 91.2 | — | — | — | — | — | |
| AFlowMulti=true, Topo=true, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 91.2 | — | — | — | — | — | |
| Qwen3-30B-A3BGroup=Larger, Configuration=Optimal serving and decoding2026.03 | 90 | — | — | — | — | — | |
| Qwen3-14BGroup=Larger, Configuration=Optimal serving and decoding2026.03 | 89.9 | — | — | — | — | — | |
| Gpt-oss-20b-highGroup=Larger, Configuration=Optimal serving and decoding2026.03 | 89.8 | — | — | — | — | — | |
| CoTMulti=false, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 89.6 | — | — | — | — | — | |
| HieraMASMulti=true, Topo=true, Role=true, Node=true, Backbone=Qwen3-80B2026.02 | 89.6 | — | — | — | — | — | |
| Full-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 88.8 | — | — | — | — | — | |
| GDesignerMulti=true, Topo=true, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 88.8 | — | — | — | — | — | |
| MASRouterMulti=true, Topo=true, Role=true, Node=false, Backbone=GPT-5-Mini2026.02 | 88.33 | — | — | — | — | — | |
| Random-GraphMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 88 | — | — | — | — | — | |
| Self-ConsistencyMulti=true, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 83.2 | — | — | — | — | — | |
| BaseMulti=false, Topo=false, Role=false, Node=false, Backbone=Qwen3-80B2026.02 | 82.4 | — | — | — | — | — | |
| MASRouterMulti=true, Topo=true, Role=true, Node=false, Backbone=Qwen3-80B2026.02 | 81.67 | — | — | — | — | — | |
| LCO-KLDBackbone=Qwen-3-4B2026.03 | 76.71 | — | — | — | — | — | |
| EngGPT2-16B-A3BGroup=Comparable, Configuration=Optimal serving and decoding2026.03 | 75.5 | — | — | — | — | — | |
| gemma-3-12b-itGroup=Larger, Configuration=Optimal serving and decoding2026.03 | 75.2 | — | — | — | — | — | |
| gemma-2-9b-itGroup=Comparable, Configuration=Optimal serving and decoding2026.03 | 74.8 | — | — | — | — | — | |
| LCO-MSEBackbone=Qwen-3-4B2026.03 | 74.21 | — | — | — | — | — | |
| LCO-LCHBackbone=Qwen-3-4B2026.03 | 74.12 | — | — | — | — | — | |
| PPOBackbone=Qwen-3-4B2026.03 | 73.93 | — | — | — | — | — | |
| GSPOBackbone=Qwen-3-4B2026.03 | 72.72 | — | — | — | — | — | |
| GRPOBackbone=Qwen-3-4B2026.03 | 71.35 | — | — | — | — | — | |
| DAPOBackbone=Qwen-3-4B2026.03 | 70.37 | — | — | — | — | — | |
| Moonlight-16B-A3B-InstructGroup=Comparable, Configuration=Optimal serving and decoding2026.03 | 70.1 | — | — | — | — | — | |
| Llama-3.1-8B-InstructGroup=Comparable, Configuration=Optimal serving and decoding2026.03 | 68.4 | — | — | — | — | — | |
| NGMModel=Qwen3-VL-2B2026.05 | 67.82 | — | — | — | — | — | |
| Qwen3-VL-2BModel=Qwen3-VL-2B2026.05 | 67.79 | — | — | — | — | — | |
| GRPO BaselineCondition=Baseline, Decoding strategy=greedy@12026.05 | 62.4 | 0.016 | — | 98.4 | 0.991 | 4,973 | |
| ProFILCondition=ProFIL, Decoding strategy=greedy@12026.05 | 62.4 | 0.008 | 50 | 99.2 | — | 5,240 | |
| LCO-LCHBackbone=Qwen-2.5-3B2026.03 | 58.71 | — | — | — | — | — | |
| LCO-KLDBackbone=Qwen-2.5-3B2026.03 | 57.58 | — | — | — | — | — | |
| GRPOBackbone=Qwen-2.5-3B2026.03 | 56.97 | — | — | — | — | — | |
| LCO-MSEBackbone=Qwen-2.5-3B2026.03 | 56.73 | — | — | — | — | — | |
| GSPOBackbone=Qwen-2.5-3B2026.03 | 56.44 | — | — | — | — | — | |
| PPOBackbone=Qwen-2.5-3B2026.03 | 55.59 | — | — | — | — | — | |
| DAPOBackbone=Qwen-2.5-3B2026.03 | 52.49 | — | — | — | — | — |