Code Generation on MBPP (Accuracy, AVG, Improvement Overhead)
81.2AccuracyQwen3-235B-A22B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Qwen3-235B-A22BFramework=Reference Model, Backbone Model=Qwen3-235B-A22B2025.08 | 81.2 | 78.22 | — | |
| COCO Qwen3-8B with coco(Qwen3-8B)Framework=COCO Framework (Proposed), Backbone Model=Qwen3-8B, Monitor Model=Qwen3-8B2025.08 | 76.6 | 74.18 | 6.2 | |
| COCO Qwen3-8B with coco(Llama-3.1-8B)Framework=COCO Framework (Proposed), Backbone Model=Qwen3-8B, Monitor Model=Llama-3.1-8B2025.08 | 74.4 | 74.37 | 6.5 | |
| Qwen3-8BFramework=Reference Model, Backbone Model=Qwen3-8B2025.08 | 66 | 68.52 | — | |
| Aflow-Qwen3-8BFramework=Multi-Agent Framework Baseline, Backbone Model=Qwen3-8B2025.08 | 65 | 69.86 | — | |
| COCO Llama-3.1-8B with coco(Qwen3-8B)Framework=COCO Framework (Proposed), Backbone Model=Llama-3.1-8B, Monitor Model=Qwen3-8B2025.08 | 60.4 | 63.59 | 9.5 | |
| Llama-3.1-8BFramework=Reference Model, Backbone Model=Llama-3.1-8B2025.08 | 60.2 | 55.48 | — | |
| Aflow-Llama3.1-8BFramework=Multi-Agent Framework Baseline, Backbone Model=Llama-3.1-8B2025.08 | 55.8 | 58.09 | — | |
| COCO Llama-3.1-8B with coco(Llama-3.1-8B)Framework=COCO Framework (Proposed), Backbone Model=Llama-3.1-8B, Monitor Model=Llama-3.1-8B2025.08 | 55.4 | 58.46 | 0.63 |