Code Generation on HumanEval+ (Acc., Token, Speed)
4,062Generation SpeedTextMAS
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| TextMASBackbone=Qwen3-14B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.11 | 4,062 | 81.1 | 5,934 | |
| TextMASBackbone=Qwen3-14B, MAS Setting=Hierarchical2025.11 | 3,988 | 84.1 | 8,114 | |
| TextMASBackbone=Qwen3-8B, MAS Setting=Hierarchical2025.11 | 1,809 | 76.8 | 8,768 | |
| TextMASBackbone=Qwen3-8B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.11 | 1,619 | 80.5 | 4,593 | |
| LatentMASBackbone=Qwen3-14B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.11 | 1,285 | 86.5 | 2,042 | |
| LatentMASBackbone=Qwen3-14B, MAS Setting=Hierarchical2025.11 | 1,188 | 86.6 | 1,512 | |
| MBD-LLaDA2-Mini-DMaxHardware=two H100 GPUs, Tensor Parallelism=TP=2, Decoding=single-sample2026.06 | 1,124.43 | — | — | |
| SingleBackbone=Qwen3-14B, Collaboration Method=Single-model, MAS Setting=Sequential2025.11 | 1,084 | 76.8 | 2,366 | |
| SingleBackbone=Qwen3-14B, MAS Setting=Hierarchical2025.11 | 1,084 | 76.8 | 2,366 | |
| TextMASBackbone=Qwen3-4B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.11 | 1,044 | 79.7 | 5,987 | |
| MBD-LLaDA2-MiniHardware=two H100 GPUs, Tensor Parallelism=TP=2, Decoding=single-sample2026.06 | 941.18 | — | — | |
| LLaDA2-Mini-DMaxHardware=two H100 GPUs, Tensor Parallelism=TP=2, Decoding=single-sample2026.06 | 931.55 | — | — | |
| TextMASBackbone=Qwen3-4B, MAS Setting=Hierarchical2025.11 | 931 | 76.2 | 8,127 | |
| LLaDA2-MiniHardware=two H100 GPUs, Tensor Parallelism=TP=2, Decoding=single-sample2026.06 | 824.94 | — | — | |
| SingleBackbone=Qwen3-8B, Collaboration Method=Single-model, MAS Setting=Sequential2025.11 | 502 | 74.4 | 2,507 | |
| SingleBackbone=Qwen3-8B, MAS Setting=Hierarchical2025.11 | 502 | 74.4 | 2,507 | |
| LatentMASBackbone=Qwen3-8B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.11 | 497 | 80.5 | 1,866 | |
| LatentMASBackbone=Qwen3-8B, MAS Setting=Hierarchical2025.11 | 439 | 78 | 1,274 | |
| LatentMASBackbone=Qwen3-4B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.11 | 350 | 79.9 | 1,775 | |
| LatentMASBackbone=Qwen3-4B, MAS Setting=Hierarchical2025.11 | 333 | 79.3 | 1,373 | |
| TextMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.11 | 333 | 55.4 | 740 | |
| SingleBackbone=Qwen3-4B, Collaboration Method=Single-model, MAS Setting=Sequential2025.11 | 274 | 75 | 2,380 | |
| SingleBackbone=Qwen3-4B, MAS Setting=Hierarchical2025.11 | 274 | 75 | 2,380 | |
| TextMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.11 | 199 | 43.2 | 1,042 | |
| TextMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.11 | 156 | 54.2 | 414 | |
| TextMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.11 | 133 | 43.9 | 719 | |
| LatentMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.11 | 101 | 56.7 | 218 | |
| LatentMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.11 | 88 | 55.4 | 187 | |
| SingleBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.11 | 80.6 | 51.2 | 161 | |
| SingleBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.11 | 80.6 | 51.2 | 161 | |
| LatentMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.11 | 77 | 44.5 | 338 | |
| LatentMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.11 | 72 | 45.7 | 226 | |
| SingleBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.11 | 64 | 39.6 | 227 | |
| SingleBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.11 | 64 | 39.6 | 227 |