Multi-task Knowledge and Reasoning on MMLU-Pro
74.04Average Score @1ERSS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ERSSModel=Phi-4-Reasoning2026.06 | 74.04 | — | |
| STAR-1-mixModel=Qwen3-14B, Setting=STAR-1-mix2026.02 | 73.85 | — | |
| BCModel=Qwen3-14B, Setting=BC2026.02 | 72.83 | — | |
| AgentModel=Qwen3-14B, Setting=Agent2026.02 | 72.67 | — | |
| LRSModel=Phi-42026.06 | 72.55 | — | |
| STAR-1Model=Qwen3-14B, Setting=STAR-12026.02 | 72.47 | — | |
| ALRRModel=Phi-42026.06 | 72.39 | — | |
| SafeChainModel=Qwen3-14B, Setting=SafeChain2026.02 | 72.26 | — | |
| ERSSModel=Phi-42026.06 | 72.17 | — | |
| ALRR+ERSSModel=Qwen3-4B2026.06 | 71.9 | — | |
| LRSModel=Phi-4-Reasoning2026.06 | 71.83 | — | |
| ALRR+LRSModel=Qwen3-4B2026.06 | 71.78 | — | |
| ALRRModel=Qwen3-4B2026.06 | 71.66 | — | |
| LRSModel=Qwen3-4B2026.06 | 71.41 | — | |
| ERSSModel=Qwen3-4B2026.06 | 71.15 | — | |
| IFPOModel=Qwen3-14B, Setting=IFPO2026.02 | 70.79 | — | |
| ALRRModel=Phi-4-Reasoning2026.06 | 70.59 | — | |
| STAR-1-mixModel=Qwen3-8B, Setting=STAR-1-mix2026.02 | 70.2 | — | |
| BaselineModel=Phi-4-Reasoning2026.06 | 69.62 | — | |
| BaselineModel=Phi-42026.06 | 69.55 | — | |
| STAR-1Model=Qwen3-8B, Setting=STAR-12026.02 | 69.46 | — | |
| BCModel=Qwen3-8B, Setting=BC2026.02 | 69.45 | — | |
| AgentModel=Qwen3-8B, Setting=Agent2026.02 | 69.07 | — | |
| IFPOModel=Qwen3-8B, Setting=IFPO2026.02 | 68.99 | — | |
| BaselineModel=Qwen3-4B2026.06 | 67.92 | — | |
| ERSS+LRSModel=Qwen3-4B2026.06 | 67.64 | — | |
| SafeChainModel=Qwen3-8B, Setting=SafeChain2026.02 | 67.54 | — | |
| FlowBankExecutor LLM=GPT-4o mini, Optimizer LLM=Qwen3-8B2026.06 | 67.4 | 1.52 | |
| STAR-1-mixModel=Qwen3-4B, Setting=STAR-1-mix2026.02 | 66.51 | — | |
| AFlow (GPT-4o)Executor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 65.61 | 0.89 | |
| AgentModel=Qwen3-4B, Setting=Agent2026.02 | 64.73 | — | |
| SafeChainModel=Qwen3-4B, Setting=SafeChain2026.02 | 64.72 | — | |
| AFlow (Qwen3-8B)Executor LLM=GPT-4o mini, Optimizer LLM=Qwen3-8B2026.06 | 64.62 | 1.47 | |
| STAR-1Model=Qwen3-4B, Setting=STAR-12026.02 | 64.58 | — | |
| ScoreFlowExecutor LLM=GPT-4o mini, Generator LLM=Qwen3-8B2026.06 | 64.58 | 2.62 | |
| ComplexCoTExecutor LLM=GPT-4o mini2026.06 | 64.03 | 0.35 | |
| AgentSquareExecutor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 63.75 | 0.75 | |
| BCModel=Qwen3-4B, Setting=BC2026.02 | 63.64 | — | |
| IFPOModel=Qwen3-4B, Setting=IFPO2026.02 | 63.64 | — | |
| MaASExecutor LLM=GPT-4o mini2026.06 | 62.31 | 1.29 | |
| MultiPersonaExecutor LLM=GPT-4o mini2026.06 | 61.7 | 0.36 | |
| ERSSModel=Gemma3-12B-it2026.06 | 61.29 | — | |
| ALRRModel=Gemma3-12B-it2026.06 | 61.01 | — | |
| LRSModel=Gemma3-12B-it2026.06 | 60.9 | — | |
| ERSSModel=R1-Distilled-LLaMA-8B2026.06 | 60.67 | — | |
| BaselineModel=Gemma3-12B-it2026.06 | 60.59 | — | |
| ALRRModel=R1-Distilled-LLaMA-8B2026.06 | 60.47 | — | |
| LRSModel=R1-Distilled-LLaMA-8B2026.06 | 60.31 | — | |
| ALRR+ERSSModel=R1-Distilled-LLaMA-8B2026.06 | 60.23 | — | |
| ADASExecutor LLM=GPT-4o mini, Optimizer LLM=GPT-4o2026.06 | 59.71 | 1.05 | |
| ALRR+LRSModel=R1-Distilled-LLaMA-8B2026.06 | 59.32 | — | |
| BaselineModel=R1-Distilled-LLaMA-8B2026.06 | 59.27 | — | |
| GPTSwarmExecutor LLM=GPT-4o mini2026.06 | 58.37 | 1.14 | |
| ERSS+LRSModel=R1-Distilled-LLaMA-8B2026.06 | 58.04 | — | |
| Multi-agent DebateExecutor LLM=GPT-4o mini2026.06 | 57.56 | 0.88 | |
| MedPromptExecutor LLM=GPT-4o mini2026.06 | 52.79 | 0.63 | |
| CoTExecutor LLM=GPT-4o mini2026.06 | 51.35 | 0.04 | |
| Self-ConsistencyExecutor LLM=GPT-4o mini2026.06 | 50.93 | 0.34 | |
| IOExecutor LLM=GPT-4o mini2026.06 | 50.89 | 0.03 | |
| Self-RefineExecutor LLM=GPT-4o mini2026.06 | 43.75 | 0.46 | |
| ALRRModel=Gemma3-4B-it2026.06 | 43.41 | — | |
| LRSModel=Gemma3-4B-it2026.06 | 43.32 | — | |
| ERSSModel=Gemma3-4B-it2026.06 | 42.98 | — | |
| BaselineModel=Gemma3-4B-it2026.06 | 42.73 | — | |
| GRPOBackbone=Qwen2.5-Math-7B2025.10 | 32.3 | — | |
| SePTBackbone=Qwen2.5-Math-7B2025.10 | 32.2 | — | |
| BaseBackbone=Qwen2.5-Math-7B2025.10 | 32.1 | — |