Code Generation on MBPP+
85.75AccuracyG-Memory
Evaluation Results
| Method | Links | |
|---|---|---|
| G-MemoryBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 85.75 | |
| G-MemoryBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 85.23 | |
| DECENTMEMBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 84.74 | |
| ChatDevBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 83.04 | |
| G-MemoryBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 82.45 | |
| DECENTMEMBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 82.39 | |
| DECENTMEMBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 82.27 | |
| Text-SerializationBackbone=Qwen3-14B, Workers=32026.06 | 81.75 | |
| DECENTMEMBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 81.49 | |
| G-MemoryBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 81.17 | |
| Parallel-SynthesisBackbone=Qwen3-14B, Workers=3, Merging=Weighted averaging (lambda=0.5)2026.06 | 80.42 | |
| ChatDevBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 80.36 | |
| DECENTMEMBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 79.39 | |
| VotingBackbone=Qwen3-14B, Workers=32026.06 | 79.37 | |
| ChatDevBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 79.17 | |
| DiffMASBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 78.5 | |
| G-MemoryBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 78.39 | |
| KVLINKBackbone=Qwen3-14B, Workers=32026.06 | 78.31 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 77.93 | |
| MetaGPTBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 77.69 | |
| DiffMASBackbone=Qwen3-14B2026.04 | 77.2 | |
| DECENTMEMBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 77.17 | |
| MetaGPTBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 76.95 | |
| ChatDevBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 76.78 | |
| MetaGPTBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 76.37 | |
| SingleBackbone=Qwen3-14B, Workers=12026.06 | 76.19 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 75.97 | |
| PlanningModel=Qwen3-8B2026.02 | 75.9 | |
| Primitives-based MASModel=Qwen3-8B2026.02 | 75.9 | |
| LatentMASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 75.7 | |
| Primitives-based MASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 75.7 | |
| LatentMASBackbone=Qwen3-14B2026.04 | 75.7 | |
| LatentMASBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 75.7 | |
| No memoryBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 75.52 | |
| ChatDevBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 75.37 | |
| DiffMASModel=Qwen3-8B, Communication Category=Trained Latent Communication2026.04 | 74.8 | |
| MetaGPTBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 74.69 | |
| LatentMASModel=Qwen3-8B2026.02 | 74.6 | |
| TextMASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 74.6 | |
| VotingModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 74.6 | |
| LatentMASModel=Qwen3-8B, Communication Category=Training-free Latent Communication2026.04 | 74.6 | |
| TextMASBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 74.6 | |
| G-MemoryBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 74.49 | |
| MetaGPTBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 74.49 | |
| VotingModel=Qwen3-8B2026.02 | 74.3 | |
| PlanningModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 74.3 | |
| No memoryBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 74.21 | |
| MicroMixModel=32B2025.08 | 74.1 | |
| G-MemoryBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 73.92 | |
| SingleModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 73.8 | |
| ReviewModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 73.8 | |
| SingleBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 73.8 | |
| INT8Model=32B2025.08 | 73.5 | |
| Disagreement-Guided Strategy Routing (Qwen2.5-Coder-7B-Instruct)Backbone=Qwen2.5-Coder-7B-Instruct, Sampling strategy=Ours2026.04 | 73.5 | |
| INT8Model=14B2025.08 | 73 | |
| Qwen2.5-Coder-7B-Instruct (BoN)Backbone=Qwen2.5-Coder-7B-Instruct, Sampling strategy=BoN2026.04 | 73 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 72.99 | |
| TextMASBackbone=Qwen3-14B2026.04 | 72.8 | |
| No memoryBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 72.49 | |
| ChatDevBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 72.18 | |
| No memoryBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 72.1 | |
| ExPODistillation Setting=Multi-Teacher Distillation2026.02 | 72 | |
| DiffMASModel=Qwen3-4B, Communication Category=Trained Latent Communication2026.04 | 71.9 | |
| G-MemoryBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 71.29 | |
| DiffMASBackbone=Mistral3-8B2026.04 | 71.1 | |
| ChatDevBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 71.05 | |
| G-MemoryBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 70.94 | |
| ChatDevBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 70.94 | |
| FP16Model=32B2025.08 | 70.9 | |
| ExOPDDistillation Setting=Single-Teacher Distillation2026.02 | 70.7 | |
| ExOPDDistillation Setting=Multi-Teacher Distillation2026.02 | 70.6 | |
| ReviewModel=Qwen3-8B2026.02 | 70.6 | |
| Primitives-based MASModel=DeepSeek-R1-Distill Llama-70B2026.02 | 70.6 | |
| LatentMASModel=Qwen3-4B, Communication Category=Training-free Latent Communication2026.04 | 70.6 | |
| BinaryModel=Dream-Coder-7B (instruct), NFE budget=1/1, FT steps=-2025.10 | 70.4 | |
| No memoryBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 70.23 | |
| TeacherDistillation Setting=Baseline2026.02 | 70.2 | |
| ExPODistillation Setting=Single-Teacher Distillation2026.02 | 70.2 | |
| MicroMixModel=14B2025.08 | 70.1 | |
| Qwen2.5-Coder-7B-InstructBackbone=Qwen2.5-Coder-7B-Instruct, Sampling strategy=Base2026.04 | 70.1 | |
| OPDDistillation Setting=Single-Teacher Distillation2026.02 | 69.9 | |
| G-MemoryBackbone=GEMMA4-E2B, Framework=DyLAN2026.05 | 69.89 | |
| SingleBackbone=Mistral3-8B2026.04 | 69.8 | |
| SFTDistillation Setting=Multi-Teacher Distillation2026.02 | 69.6 | |
| SMModel=Dream-Coder-7B (instruct), NFE budget=1/1, FT steps=33.5k2025.10 | 69.6 | |
| OPDDistillation Setting=Multi-Teacher Distillation2026.02 | 69.5 | |
| TextMASModel=Qwen3-8B2026.02 | 69.5 | |
| TextMASModel=Qwen3-8B, Communication Category=Text Communication2026.04 | 69.5 | |
| Qwen2.5-Coder-7B-Instruct (Majority)Backbone=Qwen2.5-Coder-7B-Instruct, Sampling strategy=Majority2026.04 | 69.5 | |
| TextMASModel=Qwen3-4B, Communication Category=Text Communication2026.04 | 69.3 | |
| DECENTMEMBackbone=GEMMA4-E2B, Framework=AutoGen2026.05 | 69.28 | |
| FP16Model=14B2025.08 | 69.2 | |
| Qwen2.5-Coder-7B-Instruct (DV)Backbone=Qwen2.5-Coder-7B-Instruct, Sampling strategy=DV2026.04 | 69.1 | |
| MetaGPTBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 68.99 | |
| MetaGPTBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 68.81 | |
| TextMASModel=DeepSeek-R1-Distill Llama-70B2026.02 | 68.8 | |
| VotingModel=DeepSeek-R1-Distill Llama-70B2026.02 | 68.8 | |
| No memoryBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 68.72 | |
| SingleBackbone=Qwen3-14B2026.04 | 68.5 | |
| Qwen2.5-Coder-7B-Instruct (SCoP)Backbone=Qwen2.5-Coder-7B-Instruct, Sampling strategy=SCoP2026.04 | 68.4 |