List your company
ResearchFormerly WizWand

Code Generation on MBPP (accuracy, speedup)

68.8AccuracyS2D2
11.39226.29641.256.104Mar 26, 2026Mar 27, 2026Mar 28, 2026Mar 29, 2026Mar 30, 2026Mar 31, 2026Apr 2, 2026
Updated 6mo ago

Evaluation results

MethodLinks
S2D2τmask=0.95, τedit=0.92026.0368.82.2
Diffusionτmask=1, τedit=0.92026.0367.60.5
Diffusionτmask=0.95, τedit=0.92026.0366.41.8
DiffusionBlock-Size=1 (AR)2026.0365.81
ARModel Scale=SDAR-8B-Chat, Decoding Strategy=AR2026.0364.41
S2D2τmask=0.7, τedit=0.52026.03642.1
S2D2 (config-A)Model Scale=SDAR-8B-Chat, Decoding Strategy=S2D2 (config-A)2026.03622.1
S2D2 (config-B)Model Scale=SDAR-8B-Chat, Decoding Strategy=S2D2 (config-B)2026.0361.42.8
Diffusion (static)Model Scale=SDAR-8B-Chat, Decoding Strategy=Diffusion (static)2026.03611.4
Diffusion (dynamic)Model Scale=SDAR-8B-Chat, Decoding Strategy=Diffusion (dynamic)2026.0360.62.1
ARModel Scale=SDAR-4B-Chat, Decoding Strategy=AR2026.0359.61
KLASS2026.0458.21.6
EntropyTokens per step=12026.04581
Top-1Tokens per step=12026.04581
Diffusionτmask=0.7, τedit=0.52026.0357.82.8
Token OrderTokens per step=12026.0457.81.5
DEMASK2026.0457.62.2
Diffusion (static)Model Scale=SDAR-4B-Chat, Decoding Strategy=Diffusion (static)2026.0357.21.5
S2D2 (config-B)Model Scale=SDAR-4B-Chat, Decoding Strategy=S2D2 (config-B)2026.03572.3
S2D2 (config-A)Model Scale=SDAR-4B-Chat, Decoding Strategy=S2D2 (config-A)2026.03561.3
Diffusion (dynamic)Model Scale=SDAR-4B-Chat, Decoding Strategy=Diffusion (dynamic)2026.0355.62.1
ARDecoding Method=Autoregressive, Block size (B)=4, Sub-block size (SB)=12026.0351.81
DiffusionDecoding Method=Diffusion, Configuration=config-A, Block size (B)=322026.0350.62.3
S2D2Decoding Method=S2D2, Configuration=config-A, Block size (B)=322026.0348.82
ARModel Scale=SDAR-1.7B-Chat, Decoding Strategy=AR2026.0348.41
DiffusionDecoding Method=Diffusion, Configuration=config-B, Block size (B)=322026.0347.82.9
DiffusionDecoding Method=Diffusion, Configuration=config-C, Block size (B)=32, Sub-block size (SB)=322026.0346.23.6
S2D2Decoding Method=S2D2, Configuration=config-C, Block size (B)=32, Sub-block size (SB)=322026.0345.63.8
S2D2 (config-A)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=S2D2 (config-A)2026.0345.41.5
S2D2Decoding Method=S2D2, Configuration=config-B, Block size (B)=322026.03453.7
S2D2 (config-B)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=S2D2 (config-B)2026.0344.42.9
EntropyTokens per step=22026.0443.81.3
Top-1Tokens per step=22026.0443.81.4
Token OrderTokens per step=22026.0440.81.5
Diffusion (dynamic)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=Diffusion (dynamic)2026.0321.61.3
Diffusion (static)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=Diffusion (static)2026.0313.61.3