List your company
ResearchFormerly WizWand

Code Generation on MBPP (Accuracy, AVG, Improvement Overhead)

81.2AccuracyQwen3-235B-A22B
54.36861.33468.375.266Aug 19, 2025
Updated 7mo ago

Evaluation results

MethodLinks
Qwen3-235B-A22BFramework=Reference Model, Backbone Model=Qwen3-235B-A22B2025.0881.278.22—
COCO Qwen3-8B with coco(Qwen3-8B)Framework=COCO Framework (Proposed), Backbone Model=Qwen3-8B, Monitor Model=Qwen3-8B2025.0876.674.186.2
COCO Qwen3-8B with coco(Llama-3.1-8B)Framework=COCO Framework (Proposed), Backbone Model=Qwen3-8B, Monitor Model=Llama-3.1-8B2025.0874.474.376.5
Qwen3-8BFramework=Reference Model, Backbone Model=Qwen3-8B2025.086668.52—
Aflow-Qwen3-8BFramework=Multi-Agent Framework Baseline, Backbone Model=Qwen3-8B2025.086569.86—
COCO Llama-3.1-8B with coco(Qwen3-8B)Framework=COCO Framework (Proposed), Backbone Model=Llama-3.1-8B, Monitor Model=Qwen3-8B2025.0860.463.599.5
Llama-3.1-8BFramework=Reference Model, Backbone Model=Llama-3.1-8B2025.0860.255.48—
Aflow-Llama3.1-8BFramework=Multi-Agent Framework Baseline, Backbone Model=Llama-3.1-8B2025.0855.858.09—
COCO Llama-3.1-8B with coco(Llama-3.1-8B)Framework=COCO Framework (Proposed), Backbone Model=Llama-3.1-8B, Monitor Model=Llama-3.1-8B2025.0855.458.460.63