Code Generation on MBPP (pass@1, Improvement Rate (%))
92.4Pass@1Uno-Orchestra
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Uno-Orchestra2026.05 | 92.4 | — | |
| AgentCoderBackbone=GPT-42023.12 | 91.8 | 34.4 | |
| AgentCoderBackbone=GPT-4-turbo2023.12 | 91.4 | 44.2 | |
| AgentCoderBackbone=GPT-3.5-turbo2023.12 | 89.9 | 72.2 | |
| CodeCoTBackbone=GPT-3.5-turbo2023.12 | 89.5 | 71.5 | |
| MetaGPTBackbone=GPT-42023.12 | 87.7 | 28.4 | |
| GPT-4on=500, Iterative Repair Setting=+VER2026.04 | 86.2 | — | |
| AgentOrchestra2026.05 | 85.7 | — | |
| xRouter2026.05 | 83.8 | — | |
| GPT-4on=500, Iterative Repair Setting=1-pass2026.04 | 83.4 | — | |
| HyEvoBackbone=gpt-4o-mini2026.03 | 83.28 | — | |
| MaASBackbone=gpt-4o-mini2026.03 | 82.17 | — | |
| AFlowBackbone=gpt-4o-mini2026.03 | 81.67 | — | |
| LFPO (All Loss)Reinforcement Learning Strategy=+ LFPO (All Loss)2026.03 | 81.6 | — | |
| AOrchestra2026.05 | 80.8 | — | |
| Self-DebuggingBackbone=GPT-42023.12 | 80.6 | 18 | |
| AGRPOReinforcement Learning Strategy=+ AGRPO2026.03 | 80.3 | — | |
| SPGReinforcement Learning Strategy=+ SPG2026.03 | 80.2 | — | |
| LFPO (Pos. Only)Reinforcement Learning Strategy=+ LFPO (Pos. Only)2026.03 | 80.1 | — | |
| Qwen-7BFT data=-, LoRA=-2026.03 | 80 | — | |
| ChatDevBackbone=GPT-42023.12 | 79.8 | 12.9 | |
| LFPO (Neg. Only)Reinforcement Learning Strategy=+ LFPO (Neg. Only)2026.03 | 79.5 | — | |
| Self-CollaborationBackbone=GPT-42023.12 | 78.9 | 15.5 | |
| WizardCoder-V1.1Size=33B, Base Model=DeepSeek-Coder2023.12 | 78.9 | — | |
| UniGRPOReinforcement Learning Strategy=+ UniGRPO2026.03 | 78.9 | — | |
| DeepSeek-Coder-instructSize=33B, Base Model=DeepSeek-Coder2023.12 | 78.7 | — | |
| coupled-GRPOReinforcement Learning Strategy=+ coupled-GRPO2026.03 | 78.6 | — | |
| AgentSquareBackbone=gpt-4o-mini2026.03 | 78.46 | — | |
| PrivCodeBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=42025.12 | 78.3 | — | |
| GPTSwarmBackbone=gpt-4o-mini2026.03 | 77.43 | — | |
| diffu-GRPOReinforcement Learning Strategy=+ diffu-GRPO2026.03 | 77.3 | — | |
| DyLANBackbone=gpt-4o-mini2026.03 | 77.3 | — | |
| ReflexionBackbone=GPT-42023.12 | 77.1 | 12.9 | |
| LLM-BlenderBackbone=gpt-4o-mini2026.03 | 77.05 | — | |
| AgentCoderBackbone=Claude-instant-12023.12 | 76.3 | 183.6 | |
| AgentCoderBackbone=PaLM Coder2023.12 | 75.9 | 135 | |
| NonDPFTBackbone=DS-Coder-6.7B, Privacy Epsilon=None2025.12 | 75.1 | — | |
| DiffuCoderReinforcement Learning Strategy=Base2026.03 | 75.1 | — | |
| AeroTherm-GPTn=500, Iterative Repair Setting=+VER2026.04 | 75 | — | |
| AgentVerseBackbone=gpt-4o-mini2026.03 | 74.28 | — | |
| Llama-3-70Bn=500, Iterative Repair Setting=+VER2026.04 | 73.8 | — | |
| SC (CoT×5)Backbone=gpt-4o-mini2026.03 | 73.6 | — | |
| MCDIFFUSEModel Type=Diffusion, Parameters=7B, Slot Ordering Strategy=MCTS2026.02 | 73.57 | — | |
| AgentVerseBackbone=GPT-42023.12 | 73.5 | 7.6 | |
| NonDPFTBackbone=CodeQwen1.5-7B, Privacy Epsilon=None2025.12 | 73.5 | — | |
| MultiPersonaBackbone=gpt-4o-mini2026.03 | 73.19 | — | |
| NonDPFTBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=None2025.12 | 72.5 | — | |
| IF-CL-MCSize=34B, Base Model=CodeLlama2023.12 | 72.4 | — | |
| ComplexCoTBackbone=gpt-4o-mini2026.03 | 72.36 | — | |
| Qwen-3BFT data=-, LoRA=-2026.03 | 72 | — | |
| AutoAgentsBackbone=gpt-4o-mini2026.03 | 71.95 | — | |
| VanillaBackbone=gpt-4o-mini2026.03 | 71.83 | — | |
| CoTBackbone=gpt-4o-mini2026.03 | 71.83 | — | |
| RAPBackbone=GPT-3.5-turbo2023.12 | 71.4 | 36.8 | |
| PrivCodeBackbone=CodeQwen1.5-7B, Privacy Epsilon=42025.12 | 70.6 | — | |
| IF-CL-MCSize=7B, Base Model=CodeLlama2023.12 | 70.4 | — | |
| LLM-DebateBackbone=gpt-4o-mini2026.03 | 70.29 | — | |
| ReflexionBackbone=GPT-3.5-turbo2023.12 | 70 | 34.1 | |
| Qwen3 8BModel Type=Autoregressive, Parameters=8B2026.02 | 69.81 | — | |
| INTERVENORBackbone=GPT-3.5-turbo2023.12 | 69.8 | 33.7 | |
| ColdStart-LLM2026.05 | 69.8 | — | |
| Llama-3-70Bn=500, Iterative Repair Setting=1-pass2026.04 | 69.6 | — | |
| CodeLlamaBackbone=CodeLlama (34B), Setting=Zero-Shot2023.12 | 69.3 | — | |
| PrivCodeBackbone=DS-Coder-6.7B, Privacy Epsilon=42025.12 | 69.3 | — | |
| IF-CL-MCSize=13B, Base Model=CodeLlama2023.12 | 69.2 | — | |
| JFTBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=42025.12 | 69.2 | — | |
| MagicoderS-CLPSize=7B, Base Model=CodeLlama2023.12 | 68.4 | — | |
| GPT-4Backbone=GPT-4, Setting=Zero-Shot2023.12 | 68.3 | — | |
| Self-CollaborationBackbone=GPT-3.5-turbo2023.12 | 68.2 | 30.7 | |
| AeroTherm-GPTn=500, Iterative Repair Setting=1-pass2026.04 | 68.2 | — | |
| ADASBackbone=gpt-4o-mini2026.03 | 68.13 | — | |
| CodeX+CodeTBackbone=CodeX (175B), Setting=Augmented2023.12 | 67.7 | — | |
| ReActBackbone=GPT-3.5-turbo2023.12 | 67 | 28.4 | |
| Few-ShotBackbone=GPT-3.5-turbo2023.12 | 65.8 | 26.1 | |
| ToTBackbone=GPT-3.5-turbo2023.12 | 65.8 | 26.1 | |
| DPFTBackbone=CodeQwen1.5-7B, Privacy Epsilon=42025.12 | 65.8 | — | |
| PrivCodeBackbone=CodeGemma-7B, Privacy Epsilon=42025.12 | 65.6 | — | |
| MacNetBackbone=gpt-4o-mini2026.03 | 65.28 | — | |
| JFTBackbone=CodeQwen1.5-7B, Privacy Epsilon=42025.12 | 65.1 | — | |
| Qwen2.5 7BModel Type=Autoregressive, Parameters=7B2026.02 | 64.9 | — | |
| NonDPFTBackbone=CodeGemma-7B, Privacy Epsilon=None2025.12 | 64 | — | |
| DP-AdapterBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=42025.12 | 63.9 | — | |
| DP-AdapterBackbone=CodeQwen1.5-7B, Privacy Epsilon=42025.12 | 63.7 | — | |
| GPT-4-turboBackbone=GPT-4-turbo, Setting=Zero-Shot2023.12 | 63.4 | — | |
| Qwen-7BFT data=OT3, LoRA=rk 642026.03 | 63 | — | |
| JFTBackbone=CodeGemma-7B, Privacy Epsilon=42025.12 | 61.7 | — | |
| JFTBackbone=DS-Coder-6.7B, Privacy Epsilon=42025.12 | 60.8 | — | |
| Self-debuggingBackbone=GPT-3.5-turbo2023.12 | 60.1 | 15.1 | |
| Qwen-7BFT data=MoT, LoRA=-2026.03 | 60 | — | |
| Qwen-7BFT data=OT3 + MoT, LoRA=rk 1282026.03 | 60 | — | |
| BaselineModel=Llama 3.1-8B2026.02 | 60 | — | |
| TurboQuantModel=Llama 3.1-8B2026.02 | 59.8 | — | |
| InnerQHybridModel=Llama 3.1-8B2026.02 | 59.8 | — | |
| InnerQBaseModel=Llama 3.1-8B2026.02 | 59.4 | — | |
| InnerQSmallModel=Llama 3.1-8B2026.02 | 59 | — | |
| DPFTBackbone=Qwen2.5-Coder-7B, Privacy Epsilon=42025.12 | 58.7 | — | |
| Self-PlaningBackbone=GPT-3.5-turbo2023.12 | 58.6 | 12.3 | |
| DP-AdapterBackbone=DS-Coder-6.7B, Privacy Epsilon=42025.12 | 58.4 | — | |
| DP-AdapterBackbone=CodeGemma-7B, Privacy Epsilon=42025.12 | 58.4 | — | |
| KIVISinkModel=Llama 3.1-8B2026.02 | 58.4 | — |