List your company
ResearchFormerly WizWand

Code Generation on HumanEval+ (Acc., Token, Speed)

4,062Generation SpeedTextMAS
-95.92983.542,0633,142.46Nov 25, 2025Dec 30, 2025Feb 4, 2026Mar 12, 2026Apr 17, 2026May 23, 2026Jun 28, 2026
Updated 4mo ago

Evaluation results

MethodLinks
TextMASBackbone=Qwen3-14B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.114,06281.15,934
TextMASBackbone=Qwen3-14B, MAS Setting=Hierarchical2025.113,98884.18,114
TextMASBackbone=Qwen3-8B, MAS Setting=Hierarchical2025.111,80976.88,768
TextMASBackbone=Qwen3-8B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.111,61980.54,593
LatentMASBackbone=Qwen3-14B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.111,28586.52,042
LatentMASBackbone=Qwen3-14B, MAS Setting=Hierarchical2025.111,18886.61,512
MBD-LLaDA2-Mini-DMaxHardware=two H100 GPUs, Tensor Parallelism=TP=2, Decoding=single-sample2026.061,124.43——
SingleBackbone=Qwen3-14B, Collaboration Method=Single-model, MAS Setting=Sequential2025.111,08476.82,366
SingleBackbone=Qwen3-14B, MAS Setting=Hierarchical2025.111,08476.82,366
TextMASBackbone=Qwen3-4B, Collaboration Method=Text-based MAS, MAS Setting=Sequential2025.111,04479.75,987
MBD-LLaDA2-MiniHardware=two H100 GPUs, Tensor Parallelism=TP=2, Decoding=single-sample2026.06941.18——
LLaDA2-Mini-DMaxHardware=two H100 GPUs, Tensor Parallelism=TP=2, Decoding=single-sample2026.06931.55——
TextMASBackbone=Qwen3-4B, MAS Setting=Hierarchical2025.1193176.28,127
LLaDA2-MiniHardware=two H100 GPUs, Tensor Parallelism=TP=2, Decoding=single-sample2026.06824.94——
SingleBackbone=Qwen3-8B, Collaboration Method=Single-model, MAS Setting=Sequential2025.1150274.42,507
SingleBackbone=Qwen3-8B, MAS Setting=Hierarchical2025.1150274.42,507
LatentMASBackbone=Qwen3-8B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.1149780.51,866
LatentMASBackbone=Qwen3-8B, MAS Setting=Hierarchical2025.11439781,274
LatentMASBackbone=Qwen3-4B, Collaboration Method=Latent-based MAS, MAS Setting=Sequential2025.1135079.91,775
LatentMASBackbone=Qwen3-4B, MAS Setting=Hierarchical2025.1133379.31,373
TextMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.1133355.4740
SingleBackbone=Qwen3-4B, Collaboration Method=Single-model, MAS Setting=Sequential2025.11274752,380
SingleBackbone=Qwen3-4B, MAS Setting=Hierarchical2025.11274752,380
TextMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.1119943.21,042
TextMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.1115654.2414
TextMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.1113343.9719
LatentMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.1110156.7218
LatentMASBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.118855.4187
SingleBackbone=Llama-3.1-8B-Instruct, MAS Setting=Sequential2025.1180.651.2161
SingleBackbone=Llama-3.1-8B-Instruct, MAS Setting=Hierarchical2025.1180.651.2161
LatentMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.117744.5338
LatentMASBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.117245.7226
SingleBackbone=Llama-3.2-3B-Instruct, MAS Setting=Sequential2025.116439.6227
SingleBackbone=Llama-3.2-3B-Instruct, MAS Setting=Hierarchical2025.116439.6227