List your company
ResearchFormerly WizWand

Code Generation on HumanEval (Pass@4 Accuracy and GRPO Speedup)

77Accuracy (Pass@4)DUET
32.844.27555.7567.225May 8, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DUETBackbone=Qwen3-4B-base, Budget=50%2026.05772.38
DUETBackbone=Qwen3-4B-base, Budget=Full2026.05771.43
DAPOBackbone=Qwen3-4B-base2026.0575.20.63
VIPBackbone=Qwen3-4B-base2026.0575.20.75
GRPOBackbone=Qwen3-4B-base2026.0573.61
ARRoLBackbone=Qwen3-4B-base2026.0573.61.19
BaselineBackbone=Qwen3-4B-base2026.0562.8—
VIPBackbone=Llama-3.2-3B-Instruct2026.05530.95
DUETBackbone=Llama-3.2-3B-Instruct, Budget=50%2026.0552.92.04
GRPOBackbone=Llama-3.2-3B-Instruct2026.0552.81
ARRoLBackbone=Llama-3.2-3B-Instruct2026.0552.81.13
DUETBackbone=Qwen3-1.7B-base, Budget=Full2026.0552.71.62
DAPOBackbone=Llama-3.2-3B-Instruct2026.0552.70.82
DUETBackbone=Llama-3.2-3B-Instruct, Budget=Full2026.0552.11.26
DUETBackbone=Qwen3-1.7B-base, Budget=50%2026.0551.72.51
BaselineBackbone=Llama-3.2-3B-Instruct2026.0550.5—
VIPBackbone=Qwen3-1.7B-base2026.0548.90.95
ARRoLBackbone=Qwen3-1.7B-base2026.0547.71.3
GRPOBackbone=Qwen3-1.7B-base2026.0547.61
DAPOBackbone=Qwen3-1.7B-base2026.0546.80.81
BaselineBackbone=Qwen3-1.7B-base2026.0534.5—