Accuracy Evaluation on BBH General Reasoning
94.6BBH General Reasoning AccuracyGPT-5 high
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-5 highInput Modality=Text, LLM-as-a-Judge=GPT-4o2025.11 | 94.6 | — | |
| Claude Sonnet 4.5Input Modality=Text, LLM-as-a-Judge=GPT-4o2025.11 | 93.8 | — | |
| AceMADBase Model=Qwen3-235B-A22B-Instruct, Iteration Rounds (T)=5, Number of Agents (N)=52026.03 | 92.07 | — | |
| AceMADBase Model=Qwen3-235B-A22B-Instruct, Iteration Rounds (T)=3, Number of Agents (N)=52026.03 | 91.03 | — | |
| Gemini 2.5 ProInput Modality=Text, LLM-as-a-Judge=GPT-4o2025.11 | 90 | — | |
| AceMADBase Model=Qwen3-235B-A22B-Instruct, Iteration Rounds (T)=2, Number of Agents (N)=52026.03 | 89.66 | — | |
| Kimi-K2 Base# Shots=3-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 88.7 | — | |
| DeepSeek-V3.2 Exp Base# Shots=3-shot, # Activated Params=37B, # Total Params=671B2026.01 | 88.7 | — | |
| MiMo-V2-Flash Base# Shots=3-shot, # Activated Params=15B, # Total Params=309B2026.01 | 88.5 | — | |
| MiMo-V2 FlashShots=3-shot, # Total Params=309B, # Active Params=15.1B2026.05 | 88.5 | — | |
| DeepSeek-V3.1 Base# Shots=3-shot, # Activated Params=37B, # Total Params=671B2026.01 | 88.2 | — | |
| GPTSwarmBackbone=DeepSeek-V3, Category=Multi-Agent2026.02 | 88 | — | |
| Qwen3.5Shots=3-shot, # Total Params=35B, # Active Params=2.6B2026.05 | 86.3 | — | |
| Symphony-CoordBackbone=DeepSeek-V3, Category=Multi-Agent2026.02 | 86 | — | |
| AFLOWBackbone=DeepSeek-V3, Category=Multi-Agent2026.02 | 84 | — | |
| Symphony-CoordBackbone=GPT-5-nano, Category=Multi-Agent2026.02 | 83.5 | — | |
| AFLOWBackbone=DeepSeek-V3-0324, Category=Multi-Agent2026.02 | 83 | — | |
| Symphony-CoordBackbone=DeepSeek-V3-0324, Category=Multi-Agent2026.02 | 81 | — | |
| GPTSwarmBackbone=GPT-5-nano, Category=Multi-Agent2026.02 | 81 | — | |
| LAGUNA XS.2Shots=3-shot, # Total Params=33.4B, # Active Params=3B2026.05 | 80.9 | — | |
| Sora-2 AudioInput Modality=Audio, LLM-as-a-Judge=GPT-4o2025.11 | 80.6 | — | |
| Nemotron-3-NanoShots=3-shot, # Total Params=31.6B, # Active Params=3.6B2026.05 | 79.1 | — | |
| GPTSwarmBackbone=DeepSeek-V3-0324, Category=Multi-Agent2026.02 | 79 | — | |
| AceMADBase Model=GPT-4o-mini, Iteration Rounds (T)=3, Number of Agents (N)=52026.03 | 78.28 | — | |
| AceMADBase Model=GPT-4o-mini, Iteration Rounds (T)=5, Number of Agents (N)=52026.03 | 77.24 | — | |
| AFLOWBackbone=GPT-5-nano, Category=Multi-Agent2026.02 | 76.5 | — | |
| Gemma-4Shots=3-shot, # Total Params=25.2B, # Active Params=3.8B2026.05 | 76.4 | — | |
| ATARBase Model=Qwen3-4B2025.10 | 75.31 | — | |
| ATARBase Model=Phi-42025.10 | 75.31 | — | |
| ATARBase Model=Qwen3-30B2025.10 | 74.43 | — | |
| CoTBase Model=Qwen3-4B2025.10 | 73.33 | — | |
| CoTBase Model=Qwen3-30B2025.10 | 72.8 | — | |
| AceMADBase Model=GPT-4o-mini, Iteration Rounds (T)=2, Number of Agents (N)=52026.03 | 72.76 | — | |
| CoTBase Model=Phi-42025.10 | 72.08 | — | |
| MetaGPTBackbone=DeepSeek-V3-0324, Category=Multi-Agent2026.02 | 72 | — | |
| Sora-2 Last FrameInput Modality=Last Frame, LLM-as-a-Judge=GPT-4o2025.11 | 69.8 | — | |
| Self-RefinementBackbone=DeepSeek-V3, Category=Single Agent2026.02 | 69.5 | — | |
| MetaGPTBackbone=GPT-5-nano, Category=Multi-Agent2026.02 | 69.5 | — | |
| Self-ConsistencyBackbone=DeepSeek-V3, Category=Single Agent2026.02 | 67.5 | — | |
| Centralized MADBase Model=Qwen3-235B-A22B-Instruct, Number of Agents (N)=52026.03 | 65.17 | — | |
| MetaGPTBackbone=DeepSeek-V3, Category=Multi-Agent2026.02 | 65 | — | |
| Self-ConsistencyBackbone=GPT-5-nano, Category=Single Agent2026.02 | 65 | — | |
| Qwen3-8BParameters=8B2026.03 | 63.8 | — | |
| Sparse MADBase Model=Qwen3-235B-A22B-Instruct, Number of Agents (N)=52026.03 | 63.1 | — | |
| ATARBase Model=Phi-4-mini2025.10 | 62.42 | — | |
| ReactBackbone=GPT-5-nano, Category=Single Agent2026.02 | 61 | — | |
| VEPO-7B (Full)Parameters=7B, Configuration=Full2026.03 | 60.9 | — | |
| CoTBase Model=Phi-4-mini2025.10 | 60.51 | — | |
| HSA-ULTraining Strategy=Annealing, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.11 | 60.11 | — | |
| VEPO-7B w/o CPTParameters=7B, CPT=excluded2026.03 | 59.7 | — | |
| VEPO-7B-SFTParameters=7B, Stage=SFT2026.03 | 59.6 | — | |
| Self-RefinementBackbone=GPT-5-nano, Category=Single Agent2026.02 | 59.5 | — | |
| ATARBase Model=Llama3.1-8B2025.10 | 58.53 | — | |
| MorphagentBackbone=DeepSeek-V3, Category=Multi-Agent2026.02 | 58 | — | |
| SynapseBackbone=DeepSeek-V3-0324, Category=Single Agent2026.02 | 57.5 | — | |
| SynapseBackbone=GPT-5-nano, Category=Single Agent2026.02 | 57.5 | — | |
| SynapseBackbone=DeepSeek-V3, Category=Single Agent2026.02 | 56.5 | — | |
| DirectBackbone=GPT-5-nano, Category=Single Agent2026.02 | 56.5 | — | |
| ReactBackbone=DeepSeek-V3, Category=Single Agent2026.02 | 56 | — | |
| Gemma3-27B-ITParameters=27B2026.03 | 55.9 | — | |
| DirectBackbone=DeepSeek-V3-0324, Category=Single Agent2026.02 | 55.5 | — | |
| Cold StartBackbone=DeepSeek-V3, Category=Multi-Agent2026.02 | 55 | — | |
| Self-ConsistencyBackbone=DeepSeek-V3-0324, Category=Single Agent2026.02 | 55 | — | |
| Qwen2.5-7B-Inst.Parameters=7B2026.03 | 54.4 | — | |
| MorphagentBackbone=DeepSeek-V3-0324, Category=Multi-Agent2026.02 | 54 | — | |
| Self-RefinementBackbone=DeepSeek-V3-0324, Category=Single Agent2026.02 | 53.5 | — | |
| Cold StartBackbone=DeepSeek-V3-0324, Category=Multi-Agent2026.02 | 53 | — | |
| DirectBackbone=DeepSeek-V3, Category=Single Agent2026.02 | 52 | — | |
| HSA-ULTraining Strategy=Base, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.11 | 51.7 | — | |
| ReactBackbone=DeepSeek-V3-0324, Category=Single Agent2026.02 | 51 | — | |
| ATARBase Model=Llama3.2-3B2025.10 | 50.48 | — | |
| TRM-MoETraining Strategy=Base, Architecture=MoE, Total Params=8B, Activated Params=1B, Training Tokens=8T2025.11 | 50.34 | — | |
| CoTBase Model=Llama3.1-8B2025.10 | 49.45 | — | |
| Apertus-8B-Inst.Parameters=8B2026.03 | 49.2 | — | |
| Sparse MADBase Model=GPT-4o-mini, Number of Agents (N)=52026.03 | 47.24 | — | |
| WavefrontDiffusionModel Family=Dream-7B2025.11 | 46.91 | 105.7 | |
| MorphagentBackbone=GPT-5-nano, Category=Multi-Agent2026.02 | 46.5 | — | |
| CAMELTraining Objective=Balanced, Sampling Strategy=Hourglass2026.03 | 45.5 | — | |
| DMLTraining Objective=Balanced, Sampling Strategy=Rectangle2026.03 | 45.4 | — | |
| SODMTraining Objective=Balanced, Sampling Strategy=Rectangle2026.03 | 45.3 | — | |
| WavefrontDiffusionModel Family=LLaDA-1.52025.11 | 45.26 | 114.1 | |
| BlockDiffusionModel Family=Dream-7B2025.11 | 45.13 | 100.5 | |
| BlockDiffusionModel Family=LLaDA-1.52025.11 | 44.56 | 118.6 | |
| WavefrontDiffusionModel Family=LLaDA-8B-Instruct2025.11 | 44.3 | 115.1 | |
| Decentralized MADBase Model=GPT-4o-mini, Number of Agents (N)=52026.03 | 44.14 | — | |
| Model-size agnosticTraining Objective=Balanced, Sampling Strategy=Rectangle2026.03 | 43.7 | — | |
| BlockDiffusionModel Family=LLaDA-8B-Instruct2025.11 | 43.23 | 116.6 | |
| Human DesignedTraining Objective=Balanced, Sampling Strategy=Rectangle2026.03 | 42.9 | — | |
| Single AgentBase Model=Qwen3-235B-A22B-Instruct, Number of Agents (N)=52026.03 | 42.07 | — | |
| Qwen3Training Strategy=Annealing, Architecture=Dense, Total Params=0.6B, Activated Params=0.6B, Training Tokens=36T2025.11 | 41.28 | — | |
| Tower-Plus-9BParameters=9B2026.03 | 40.4 | — | |
| Centralized MADBase Model=GPT-4o-mini, Number of Agents (N)=52026.03 | 39.66 | — | |
| Qwen2.5-1.5B-InstructModel scale=1.5B, Evaluation mode=0-shot2026.06 | 38.19 | — | |
| Cold StartBackbone=GPT-5-nano, Category=Multi-Agent2026.02 | 36.5 | — | |
| DLLGModel scale=1.5B, Evaluation mode=0-shot2026.06 | 36.11 | — | |
| UniTeModel scale=1.5B, Evaluation mode=0-shot2026.06 | 35.53 | — | |
| Token Maj-VotingModel scale=1.5B, Evaluation mode=0-shot2026.06 | 35.42 | — | |
| LinearModel scale=1.5B, Evaluation mode=0-shot2026.06 | 35.3 | — | |
| CoTBase Model=Llama3.2-3B2025.10 | 35.09 | — | |
| GaCModel scale=1.5B, Evaluation mode=0-shot2026.06 | 34.84 | — |