Reasoning on MMLU-Pro
92.86AccuracyAgent Q-Mix
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Agent Q-MixBase Model=GPT-OSS:120B2026.04 | 92.86 | — | |
| LobsterBase Model=Gemini-3.1-Flash-Lite, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 91.43 | — | |
| LangGraphBase Model=Gemini-3.1-Flash-Lite, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 91.43 | — | |
| Gemini-3.0 ProVariant=Pro2026.01 | 90.1 | — | |
| Agent FrameworkBase Model=Gemini-3.1-Flash-Lite, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 90 | — | |
| TopoDIMBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 88.57 | — | |
| AutoGenBase Model=Gemini-3.1-Flash-Lite, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 88.57 | — | |
| Agent Q-MixBase Model=Gemini-3.1-Flash-Lite2026.04 | 88.57 | — | |
| Claude Sonnet 4.5Variant=Sonnet 4.52026.01 | 88.2 | — | |
| GPT-5 HighVariant=High2026.01 | 87.5 | — | |
| LobsterBase Model=GPT-OSS:120B, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 87.14 | — | |
| GTDBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 87.14 | — | |
| Qwen3.5-27BArchitecture=Dense, # Total Params=27B, # Activated Params=27B, Reasoning Mode=REASONING2026.04 | 86.1 | — | |
| GTDBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 85.71 | — | |
| MaASBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 85.71 | — | |
| TopoDIMBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 85.71 | — | |
| In-placeModel=Qwen, Number of turns=92025.10 | 85.4 | — | |
| DeepSeek-V3.2 ThinkingThinking Mode=true2026.01 | 85 | — | |
| In-placeModel=Qwen, Number of turns=82025.10 | 85 | — | |
| MiMo-V2-FlashVariant=Flash2026.01 | 84.9 | — | |
| In-placeModel=Llama, Number of turns=92025.10 | 84.8 | — | |
| Kimi-K2 ThinkingThinking Mode=true2026.01 | 84.6 | — | |
| In-placeModel=Qwen, Number of turns=72025.10 | 84.6 | — | |
| G-DesignerBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 84.29 | — | |
| MaASBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 84.29 | — | |
| In-placeModel=Llama, Number of turns=82025.10 | 84.2 | — | |
| K-EXAONE-236B-A23BArchitecture=MoE, # Total Params=236B, # Activated Params=23B, Reasoning Mode=REASONING2026.04 | 83.8 | — | |
| Qwen3-VL-235B-A22BArchitecture=MoE, # Total Params=236B, # Activated Params=22B, Reasoning Mode=Thinking2026.04 | 83.8 | — | |
| In-placeModel=Qwen, Number of turns=62025.10 | 83.8 | — | |
| EXAONE 4.5 33BArchitecture=Dense, # Total Params=33B, # Activated Params=32B, Reasoning Mode=REASONING2026.04 | 83.3 | — | |
| GPT-5 miniReasoning Mode=REASONING: HIGH2026.04 | 83.3 | — | |
| In-placeModel=Qwen, Number of turns=52025.10 | 83 | — | |
| AgentDropoutBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 82.86 | — | |
| LLM-DebateBase Model=Gemini-3.1-Flash-Lite, Framework Category=Static multi-agent2026.04 | 82.86 | — | |
| GPTSwarmBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 82.86 | — | |
| In-placeModel=Llama, Number of turns=72025.10 | 82.8 | — | |
| SMCSCategory=Ours2025.07 | 82.02 | — | |
| In-placeModel=Qwen, Number of turns=42025.10 | 81.9 | — | |
| GPTSwarmBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 81.43 | — | |
| AutoGenBase Model=GPT-OSS:120B, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 81.43 | — | |
| Base (direct)Base Model=Gemini-3.1-Flash-Lite, Framework Category=Single-agent baseline2026.04 | 81.43 | — | |
| QwQ-32BCoT Pattern=Long-CoT, Training Strategy=Base, Model Series=QwQ, Model Size=32B, T=0.7, top-p=0.95, Inference Averaging=5 runs2025.09 | 81.4 | — | |
| QwQ-32B-CodeGymCoT Pattern=Long-CoT, Training Strategy=CodeGym, Model Series=QwQ, Model Size=32B, T=0.7, top-p=0.95, Inference Averaging=5 runs2025.09 | 81.4 | — | |
| In-placeModel=Gemma, Number of turns=92025.10 | 81.3 | — | |
| In-placeModel=Llama, Number of turns=62025.10 | 81.3 | — | |
| MemoryModel=Qwen, Number of turns=92025.10 | 80.9 | — | |
| Majority VotingCategory=Other Methods2025.07 | 80.85 | — | |
| MemoryModel=Qwen, Number of turns=82025.10 | 80.7 | — | |
| In-placeModel=Gemma, Number of turns=82025.10 | 80.7 | — | |
| Symbolic-MoE*Category=Other Methods2025.07 | 80.6 | — | |
| Multi-turnModel=Qwen, Number of turns=92025.10 | 80.5 | — | |
| GPT-4.1Access=Close-source2025.07 | 80.43 | — | |
| Multi-turnModel=Qwen, Number of turns=82025.10 | 80.1 | — | |
| In-placeModel=Qwen, Number of turns=32025.10 | 80.1 | — | |
| Agent FrameworkBase Model=GPT-OSS:120B, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 80 | — | |
| MemoryModel=Qwen, Number of turns=72025.10 | 79.7 | — | |
| In-placeModel=Llama, Number of turns=52025.10 | 79.7 | — | |
| MoACategory=Other Methods2025.07 | 79.6 | — | |
| In-placeModel=Gemma, Number of turns=72025.10 | 79.5 | — | |
| Multi-turnModel=Qwen, Number of turns=72025.10 | 79.3 | — | |
| Multi-turnModel=Qwen, Number of turns=62025.10 | 79.1 | — | |
| MemoryModel=Llama, Number of turns=92025.10 | 78.7 | — | |
| LLM-DebateBase Model=GPT-OSS:120B, Framework Category=Static multi-agent2026.04 | 78.57 | — | |
| G-DesignerBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 78.57 | — | |
| MemoryModel=Qwen, Number of turns=62025.10 | 78.5 | — | |
| In-placeModel=Qwen, Number of turns=22025.10 | 78.5 | — | |
| Claude-3.5-SonnetAccess=Close-source2025.07 | 78.34 | — | |
| MemoryModel=Llama, Number of turns=82025.10 | 78.3 | — | |
| FP16Backbone=Qwen3-14B, Zero-shot=true2025.11 | 78.1 | — | |
| Multi-turnModel=Qwen, Number of turns=52025.10 | 78.1 | — | |
| Self Consistency (Best on Validation)Category=Other Methods2025.07 | 78.01 | — | |
| QTIPBackbone=Qwen3-14B, Type=vector, Zero-shot=true2025.11 | 77.9 | — | |
| Qwen3-32BAccess=Open-source, Parameters=32B2025.07 | 77.76 | — | |
| Open-source Upper BoundCategory=Baselines2025.07 | 77.76 | — | |
| MemoryModel=Qwen, Number of turns=52025.10 | 77.7 | — | |
| In-placeModel=Gemma, Number of turns=62025.10 | 77.7 | — | |
| ParoQuantBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 77.5 | — | |
| Multi-turnModel=Qwen, Number of turns=42025.10 | 77.3 | — | |
| MemoryModel=Llama, Number of turns=72025.10 | 77.3 | — | |
| AWQBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 77.2 | — | |
| GLM-Z1-32B-0414Access=Open-source, Parameters=32B2025.07 | 76.76 | — | |
| E-QATBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 76.5 | — | |
| In-placeModel=Llama, Number of turns=42025.10 | 76.5 | — | |
| In-placeModel=Gemma, Number of turns=52025.10 | 76.1 | — | |
| MemoryModel=Llama, Number of turns=62025.10 | 75.9 | — | |
| MemoryModel=Qwen, Number of turns=42025.10 | 75.7 | — | |
| DeepSeek-R1-Distill-Qwen-32BAccess=Open-source, Parameters=32B2025.07 | 75.17 | — | |
| Multi-turnModel=Qwen, Number of turns=32025.10 | 75.1 | — | |
| Multi-turnModel=Llama, Number of turns=92025.10 | 75.1 | — | |
| Multi-turnModel=Llama, Number of turns=82025.10 | 74.8 | — | |
| DeepSeek-R1-Distill-Llama-70BAccess=Open-source, Parameters=70B2025.07 | 74.75 | — | |
| QwQ-32BAccess=Open-source, Parameters=32B2025.07 | 74.67 | — | |
| FP16Backbone=Qwen3-8B, Zero-shot=true2025.11 | 74.6 | — | |
| Base (direct)Base Model=GPT-OSS:120B, Framework Category=Single-agent baseline2026.04 | 74.29 | — | |
| AgentDropoutBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 74.29 | — | |
| In-placeModel=Llama, Number of turns=32025.10 | 74.2 | — | |
| HuatuoGPT-o1-72BAccess=Open-source, Parameters=72B2025.07 | 74.16 | — | |
| ParoQuantBackbone=Qwen3-8B, Type=linear, Zero-shot=true2025.11 | 74.1 | — | |
| QTIPBackbone=Qwen3-8B, Type=vector, Zero-shot=true2025.11 | 74 | — | |
| GPT-o3-miniAccess=Close-source2025.07 | 74 | — |