List your company
ResearchFormerly WizWand

Code Generation on HumanEval (Acc., TPF, TPS, Speedup vs. AR)

87.8AccuracyAR (greedy)
52.9662.00571.0580.095Dec 16, 2025
Updated 8mo ago

Evaluation results

MethodLinks
AR (greedy)Family=AR, Backbone=Qwen2.5-Coder-7B-Instruct, Hardware=single B200 GPU2025.1287.8100831
JacobiFamily=AR + Jacobi, Backbone=Qwen2.5-Coder-7B-Instruct, Hardware=single B200 GPU2025.1287.810584.71.02
CLLMFamily=AR + Jacobi, Backbone=Qwen2.5-Coder-7B-Instruct, Hardware=single B200 GPU2025.1287.8268207.42.5
Jacobi Forcing ModelFamily=AR + Jacobi, Backbone=Qwen2.5-Coder-7B-Instruct, Hardware=single B200 GPU2025.1283.5401301.653.63
Jacobi Forcing Model (MR)Family=AR + Jacobi, Backbone=Qwen2.5-Coder-7B-Instruct, Hardware=single B200 GPU, Decoding Mode=multi-block decoding with rejection recycling2025.1283.5421327.963.95
SDARFamily=dLLM, Hardware=single B200 GPU2025.1278.723631.460.38
EAGLE-3*Family=AR + Spec-Dec, Hardware=single B200 GPU2025.1268.9638246.12.97
Fast-dLLM v2Family=dLLM, Hardware=single B200 GPU2025.1263.410083.291
HASS*Family=AR + Spec-Dec, Hardware=single B200 GPU2025.1261.6553280.293.37
dParallelFamily=dLLM (distilled), Hardware=single B200 GPU2025.1254.3290175.152.11