Code Generation on MBPP (test)
95.1Pass@1AgentConductor
Evaluation results
| Method | Links | |||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AgentConductorSystem Category=Multi-Agent Systems with Topology Optimization, Model Scale=3B2026.02 | 95.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| QualityFlow# of prompts=Multiple2025.01 | 94.2 | — | — | — | — | — | — | 96.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MapCoderSystem Category=Classical Multi-Agent Systems2026.02 | 94.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FlowReasonerSystem Category=Multi-Agent Systems with Workflow Optimization2026.02 | 93.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AutoGenSystem Category=Classical Multi-Agent Systems2026.02 | 92.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MetaGPTSystem Category=Classical Multi-Agent Systems2026.02 | 92.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AgentPruneSystem Category=Multi-Agent Systems with Topology Optimization, Topology Variant=Layered2026.02 | 92.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AgentCoder# of prompts=Multiple2025.01 | 91.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AgentPruneSystem Category=Multi-Agent Systems with Topology Optimization, Topology Variant=Complex2026.02 | 91.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPTSwarmSystem Category=Multi-Agent Systems with Topology Optimization2026.02 | 91.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| G-DesignerSystem Category=Multi-Agent Systems with Topology Optimization2026.02 | 90.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MacNetSystem Category=Multi-Agent Systems with Topology Optimization, Topology Variant=Layered2026.02 | 90.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Chain-of-AgentsSystem Category=Multi-Agent Systems with Workflow Optimization, Model Scale=32B2026.02 | 90.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-Coder-V2-Instruct# of prompts=One2025.01 | 89.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MacNetSystem Category=Multi-Agent Systems with Topology Optimization, Topology Variant=Complex2026.02 | 89.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Claude Sonnet-3.5# of prompts=One, status=reproduced2025.01 | 88.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AVSPOModel=Qwen2.5-Coder-7B2026.05 | 86.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.2 | — | — | — | — | — | — | |
| Claude Opus-3# of prompts=One2025.01 | 86.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MANGOBase LLM=GPT-4o-mini2026.05 | 85.63 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PRISMBase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Proposers=3, Reviewer=1, Synthesis iterations=32026.02 | 84.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel=Qwen2.5-Coder-7B2026.05 | 83.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MANGO (Skip-2)Base LLM=GPT-4o-mini2026.05 | 83.58 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MapCoder# of prompts=Multiple2025.01 | 83.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT4-Turbo2023.12 | 83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFlowSystem Category=Multi-Agent Systems with Workflow Optimization2026.02 | 82.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoRe-CodeBased model=Q2.5-14B-CI2026.05 | 82.2 | — | — | — | — | — | — | 84.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 86.5 | — | |
| MaASBase LLM=GPT-4o-mini2026.05 | 82.17 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT3.5-Turbo2023.12 | 81.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFlowBase LLM=GPT-4o-mini2026.05 | 81.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.2Type=Single-model reference, Zero-shot prompting=true, Thinking mode=false2026.02 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOBased model=Q2.5-14B-CI2026.05 | 80.7 | — | — | — | — | — | — | 83.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 85.2 | — | |
| CoRe-CodeBased model=Q3-4B2026.05 | 80.5 | — | — | — | — | — | — | 82.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 82.2 | — | |
| CodeRL+Based model=Q2.5-14B-CI2026.05 | 80.4 | — | — | — | — | — | — | 82.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 85.5 | — | |
| Qwen3-235B-A22BType=Single-model reference, Zero-shot prompting=true, Thinking mode=false2026.02 | 80.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4Model=GPT-4, Strategy=Zero-shot, Zero-shot=true2023.03 | 80.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4Model Type=Instruction-Tuned Model, Setting=Few-shot2024.01 | 80 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOBased model=Q3-4B2026.05 | 78.5 | — | — | — | — | — | — | 80.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.8 | — | |
| CUREBased model=Q2.5-14B-CI2026.05 | 78.5 | — | — | — | — | — | — | 80.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.9 | — | |
| AgentSquareBase LLM=GPT-4o-mini2026.05 | 78.46 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Focused-DPOBased model=Q2.5-14B-CI2026.05 | 78.4 | — | — | — | — | — | — | 79.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.2 | — | |
| Q2.5-14B-CIBased model=Q2.5-14B-CI2026.05 | 78.3 | — | — | — | — | — | — | 81.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 85 | — | |
| Self-ConsistencyBase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Sampling=K=5 paths2026.02 | 78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPTSwarmBase LLM=GPT-4o-mini2026.05 | 77.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DyLANBase LLM=GPT-4o-mini2026.05 | 77.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Two HeadsBase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Solvers=K=32026.02 | 77.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ReConcileBase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Agents=3, Discussion rounds=22026.02 | 77.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Focused-DPOBased model=Q3-4B2026.05 | 77.2 | — | — | — | — | — | — | 79.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.2 | — | |
| ReflexionModel=GPT-4, Strategy=Reflexion, Zero-shot=true2023.03 | 77.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Reflexion# of prompts=Multiple2025.01 | 77.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel=Qwen2.5-3B-Inst2026.05 | 77 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseModel=Qwen2.5-Coder-7B2026.05 | 76.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoABase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Agents per layer=32026.02 | 76.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-Debugging, +Trace# of prompts=Multiple2025.01 | 76.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PPOModel=Qwen2.5-3B-Inst2026.05 | 76.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LDBModel=GPT-3.5 (>=175B), Debugger=LDB2024.02 | 76 | — | — | — | — | — | — | — | — | — | — | — | 8.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LDB# of prompts=Multiple2025.01 | 76 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-30B-A3BType=Single-model reference, Zero-shot prompting=true, Thinking mode=false2026.02 | 76 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RandOptModel=Qwen2.5-3B-Inst, K=502026.05 | 75.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CUREBased model=Q3-4B2026.05 | 75.7 | — | — | — | — | — | — | 79.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.7 | — | |
| RandOptModel=OLMo3-7B-Inst, K=502026.05 | 75.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeRL+Based model=Q3-4B2026.05 | 74.5 | — | — | — | — | — | — | 77.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.2 | — | |
| SD (+Expl.)Model=GPT-3.5 (>=175B), Debugger=SD (+Expl.)2024.02 | 74.4 | — | — | — | — | — | — | — | — | — | — | — | 6.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoRe-CodeBased model=Q2.5-7B-CI2026.05 | 73.9 | — | — | — | — | — | — | 77.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.7 | — | |
| IF-CLPSize=34B, Open-source=weight&data, Base Model=CODELLAMA-PYTHON2023.12 | 73.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SC (CoT×5)Base LLM=GPT-4o-mini2026.05 | 73.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4o-miniSystem Category=Vanilla, Backbone=GPT-4o-mini2026.02 | 73.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOBased model=Q2.5-7B-CI2026.05 | 73.4 | — | — | — | — | — | — | 75.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.5 | — | |
| MPSC-SemanticFoundation Model=GPT-3.5-Turbo2023.09 | 73.23 | — | — | — | — | 73.29 | — | 73.55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| WizardCoder-CLPSize=34B, Open-source=weight2023.12 | 73.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VEROICInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 73.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MultiPersonaBase LLM=GPT-4o-mini2026.05 | 73.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RMI 50% (G)%=502026.03 | 73.02 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Q3-4BBased model=Q3-4B2026.05 | 72.7 | — | — | — | — | — | — | 75.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 77.8 | — | |
| SD (+Trace)Model=GPT-3.5 (>=175B), Debugger=SD (+Trace)2024.02 | 72.6 | — | — | — | — | — | — | — | — | — | — | — | 5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RISK PREDICTORInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 72.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VEROICInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 72.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RISK PREDICTORInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 72.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RMI Top 50%%=502026.03 | 72.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CONFIDENCE-THRESHOLDInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 72.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CONFIDENCE-THRESHOLDInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 72 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeTFoundation Model=GPT-3.5-Turbo2023.09 | 71.9 | — | — | — | — | 71.95 | — | 72.02 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DSCV2-16BBased model=DSCV2-16B2026.05 | 71.9 | — | — | — | — | — | — | 77.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 77.9 | — | |
| IO (direct LLM invocation)Base LLM=GPT-4o-mini2026.05 | 71.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoTBase LLM=GPT-4o-mini2026.05 | 71.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OUTPUT-TRIGGERInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 71.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OUTPUT-TRIGGERInference Paths=Qwen-2.5-7B -> Qwen-2.5-32B2026.04 | 71.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IF-CLPSize=13B, Open-source=weight&data, Base Model=CODELLAMA-PYTHON2023.12 | 71.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-consistencyFoundation Model=GPT-3.5-Turbo2023.09 | 71.7 | — | — | — | — | 71.73 | — | 71.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Full Data%=1002026.03 | 71.69 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RDS+%=252026.03 | 71.69 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RMI 50-75%%=252026.03 | 71.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IF-CLSize=34B, Open-source=weight&data, Base Model=CODELLAMA2023.12 | 71.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-42023.09 | 71.26 | — | — | — | — | 74.27 | — | 76.99 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-3.5-TurboModel Type=Instruction-Tuned Model, Setting=Few-shot2024.01 | 70.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOModel=OLMo3-7B-Inst2026.05 | 70.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MBR-EXECFoundation Model=GPT-3.5-Turbo2023.09 | 70.79 | — | — | — | — | 73.14 | — | 74.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RMI QW 50%%=502026.03 | 70.63 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SCAR%=252026.03 | 70.63 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PERIODICInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 70.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RANDOMInference Paths=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 70.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |