Knowledge on MMLU-Pro
80.4ScoreQwen3-Omni 30B-A3B Thinking
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3-Omni 30B-A3B ThinkingContext Length=64K2026.07 | 80.4 | — | |
| Qwen3.5-Omni Flash 35B-A3BContext Length=256K2026.07 | 79.9 | — | |
| Qwen3.5-4BActive / Total Parameters=4.0B / 4.0B, Sampling Settings=Recommended sampling settings2026.05 | 79.7 | — | |
| Audex 30B-A3BContext Length=1M2026.07 | 78.9 | — | |
| Ling-flash-2.02026.02 | 77.55 | 100 | |
| LLaDA2.1-flashInference Mode=Q Mode2026.02 | 76.59 | 262 | |
| LLaDA2.1-flashInference Mode=S Mode2026.02 | 75.31 | 443 | |
| Step-Audio R1.1 33BContext Length=64K2026.07 | 75.3 | — | |
| LLaDA2.0-flash2026.02 | 74.79 | 236 | |
| Qwen3-4B-Thinking-2507Active / Total Parameters=4.0B / 4.0B, Sampling Settings=Recommended sampling settings2026.05 | 74.3 | — | |
| Qwen3-30B-A3B-Inst-25072026.02 | 74.21 | 100 | |
| ZAYA1-8BActive / Total Parameters=0.7B / 8.0B, Sampling Settings=T=1.0, top-p=0.95, top-k disabled2026.05 | 74.2 | — | |
| DebateBackbone=Qwen-2.5-72B-Instruct2025.05 | 74 | — | |
| DebateBackbone=Llama-3.3-70B-Instruct2025.05 | 73.6 | — | |
| Qwen3# Param.=8B2026.02 | 73.26 | — | |
| SCBackbone=Qwen-2.5-72B-Instruct2025.05 | 73 | — | |
| MiniCPM-4.1# Param.=8B2026.02 | 72.7 | — | |
| MAS-GPTBackbone=Qwen-2.5-72B-Instruct2025.05 | 72.6 | — | |
| VP2OContext Length=16K2026.06 | 72.5 | — | |
| AgentVerseBackbone=Qwen-2.5-72B-Instruct2025.05 | 72 | — | |
| Nemotron-Nano-v2# Param.=9B2026.02 | 71.79 | — | |
| VP2OContext Length=8K2026.06 | 71.5 | — | |
| GRPO-baselineContext Length=16K2026.06 | 71.4 | — | |
| MAS-GPTBackbone=Llama-3.3-70B-Instruct2025.05 | 71.2 | — | |
| CoTBackbone=Qwen-2.5-72B-Instruct2025.05 | 71.2 | — | |
| DyLANBackbone=Qwen-2.5-72B-Instruct2025.05 | 71.2 | — | |
| GRPO-baselineContext Length=8K2026.06 | 71.1 | — | |
| Falcon-H1R# Param.=7B2026.02 | 70.98 | — | |
| SCBackbone=Llama-3.3-70B-Instruct2025.05 | 70.6 | — | |
| AgentVerseBackbone=Llama-3.3-70B-Instruct2025.05 | 70.4 | — | |
| DyLANBackbone=Llama-3.3-70B-Instruct2025.05 | 70.4 | — | |
| Gemma-4-E4B-itActive / Total Parameters=4.0B / 8.0B, Sampling Settings=Recommended sampling settings2026.05 | 70.2 | — | |
| AutoGenBackbone=Qwen-2.5-72B-Instruct2025.05 | 70.2 | — | |
| CoTBackbone=Llama-3.3-70B-Instruct2025.05 | 69.8 | — | |
| MADBackbone=Llama-3.3-70B-Instruct2025.05 | 69.8 | — | |
| SingleBackbone=Qwen-2.5-72B-Instruct2025.05 | 69.8 | — | |
| AFlow-MathBackbone=Qwen-2.5-72B-Instruct2025.05 | 69.2 | — | |
| Ministral-3-R# Param.=8B2026.02 | 68.75 | — | |
| AFlow-MathBackbone=Llama-3.3-70B-Instruct2025.05 | 68.6 | — | |
| Ling-mini-2.02026.02 | 67.18 | — | |
| MiniCPM-SALA# Param.=9B2026.02 | 67.04 | — | |
| SingleBackbone=Llama-3.3-70B-Instruct2025.05 | 66.8 | — | |
| AutoGenBackbone=Llama-3.3-70B-Instruct2025.05 | 66 | — | |
| Qwen3-8Bno think=true2026.02 | 65.83 | — | |
| MADBackbone=Qwen-2.5-72B-Instruct2025.05 | 65.6 | — | |
| MacNetBackbone=Qwen-2.5-72B-Instruct2025.05 | 65.4 | — | |
| LLaDA2.1-minimode=Q Mode2026.02 | 64.84 | 2.41 | |
| LLaDA2.0-mini2026.02 | 64.27 | 2.15 | |
| MacNetBackbone=Llama-3.3-70B-Instruct2025.05 | 64 | — | |
| LLaDA2.1-minimode=S Mode2026.02 | 63.42 | 4.22 | |
| MAVBackbone=Qwen-2.5-72B-Instruct2025.05 | 61.2 | — | |
| MAVBackbone=Llama-3.3-70B-Instruct2025.05 | 61 | — | |
| Voxtral Small-24B 2507Context Length=128K2026.07 | 60.5 | — | |
| T2MBackbone=LLaDA2.1-mini, Inference Strategy=Token-to-Mask, Remasking Strategy=LOWPROB, τ=0.3, Cmax=1, ρmax=0.252026.04 | 58.84 | — | |
| Original (T2T)Backbone=LLaDA2.1-mini, Inference Strategy=Text-to-Text2026.04 | 58.78 | — | |
| Audex 2BContext Length=128K2026.07 | 57 | — | |
| MiMo-Audio 7BContext Length=8K2026.07 | 55.3 | — | |
| Qwen3-4BTotal Parameters=4B, Active Parameters=4B, Trained Tokens=36T2025.11 | 52.31 | — | |
| Qwen3-1.7B# Total Params=1.7B, # Trained Tokens=36T2025.11 | 43.04 | — | |
| DSRMidtraining Schedule=DSR2026.05 | 41.89 | — | |
| WSDMidtraining Schedule=WSD2026.05 | 40.96 | — | |
| Cosine-decayMidtraining Schedule=Cos2026.05 | 40.04 | — | |
| Unified 15B MoEMidtraining Corpus=1k→8k2026.05 | 39.07 | — | |
| Unified 15B MoEMidtraining Corpus=4k2026.05 | 38.11 | — | |
| Unified 15B MoEMidtraining Corpus=4k→8k2026.05 | 37.46 | — | |
| LFM2-8B-A1BTotal Parameters=8.3B, Active Parameters=1.5B, Trained Tokens=13T2025.11 | 37.42 | — | |
| Gemma-3-4BTotal Parameters=4B, Active Parameters=4B, Trained Tokens=4T2025.11 | 34.76 | — | |
| Granite-4.0-HTotal Parameters=7B, Active Parameters=1B, Trained Tokens=15T2025.11 | 32.03 | — | |
| UM-190kBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 31.98 | — | |
| UM-187kBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 31.32 | — | |
| TuluDPOBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 30.66 | — | |
| UM-170kBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 30.61 | — | |
| ORPOBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 30.33 | — | |
| UltraFBBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 29.99 | — | |
| SFTBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 29.38 | — | |
| HelpSteerBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 28.54 | — | |
| CodePrefBackbone=Apertus-8B-SFT, Evaluation Protocol=5-shot2025.11 | 28.34 | — | |
| Qwen3-0.6B# Total Params=0.6B, # Trained Tokens=36T2025.11 | 27.02 | — | |
| Unified 15B MoEPretraining Corpus=1k→8k2026.05 | 26.29 | — | |
| LFM2-2.6BTotal Parameters=2.6B, Active Parameters=2.6B, Trained Tokens=11T2025.11 | 25.96 | — | |
| Unified 15B MoEPretraining Corpus=4k→8k2026.05 | 24.21 | — | |
| SmolLM3-3BTotal Parameters=3.1B, Active Parameters=3.1B, Trained Tokens=11T2025.11 | 23.9 | — | |
| Unified 15B MoEPretraining Corpus=4k2026.05 | 23.61 | — | |
| Llama-3.2-3BTotal Parameters=3.2B, Active Parameters=3.2B, Trained Tokens=9T2025.11 | 22.25 | — | |
| UM-190kshots=5-shot2025.11 | 20.87 | — | |
| UM-187kshots=5-shot2025.11 | 20.5 | — | |
| TuluDPOshots=5-shot2025.11 | 19.76 | — | |
| LFM2-1.2B# Total Params=1.2B, # Trained Tokens=11T2025.11 | 19.71 | — | |
| ORPOshots=5-shot2025.11 | 19.44 | — | |
| Llama-3.2-1B# Total Params=1.2B, # Trained Tokens=9T2025.11 | 19.37 | — | |
| UM-170kshots=5-shot2025.11 | 19.31 | — | |
| UltraFBshots=5-shot2025.11 | 18.98 | — | |
| HelpSteershots=5-shot2025.11 | 18.81 | — | |
| LFM2-700M# Total Params=0.70B, # Trained Tokens=11T2025.11 | 18.65 | — | |
| SFTshots=5-shot2025.11 | 18.54 | — | |
| CodePrefshots=5-shot2025.11 | 18.27 | — | |
| LFM2-350M# Total Params=0.35B, # Trained Tokens=11T2025.11 | 15.85 | — | |
| Gemma-3-1B# Total Params=1B, # Trained Tokens=2T2025.11 | 13.72 | — |