Code Generation on MBPP (accuracy, speedup)
68.8AccuracyS2D2
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| S2D2τmask=0.95, τedit=0.92026.03 | 68.8 | 2.2 | |
| Diffusionτmask=1, τedit=0.92026.03 | 67.6 | 0.5 | |
| Diffusionτmask=0.95, τedit=0.92026.03 | 66.4 | 1.8 | |
| DiffusionBlock-Size=1 (AR)2026.03 | 65.8 | 1 | |
| ARModel Scale=SDAR-8B-Chat, Decoding Strategy=AR2026.03 | 64.4 | 1 | |
| S2D2τmask=0.7, τedit=0.52026.03 | 64 | 2.1 | |
| S2D2 (config-A)Model Scale=SDAR-8B-Chat, Decoding Strategy=S2D2 (config-A)2026.03 | 62 | 2.1 | |
| S2D2 (config-B)Model Scale=SDAR-8B-Chat, Decoding Strategy=S2D2 (config-B)2026.03 | 61.4 | 2.8 | |
| Diffusion (static)Model Scale=SDAR-8B-Chat, Decoding Strategy=Diffusion (static)2026.03 | 61 | 1.4 | |
| Diffusion (dynamic)Model Scale=SDAR-8B-Chat, Decoding Strategy=Diffusion (dynamic)2026.03 | 60.6 | 2.1 | |
| ARModel Scale=SDAR-4B-Chat, Decoding Strategy=AR2026.03 | 59.6 | 1 | |
| KLASS2026.04 | 58.2 | 1.6 | |
| EntropyTokens per step=12026.04 | 58 | 1 | |
| Top-1Tokens per step=12026.04 | 58 | 1 | |
| Diffusionτmask=0.7, τedit=0.52026.03 | 57.8 | 2.8 | |
| Token OrderTokens per step=12026.04 | 57.8 | 1.5 | |
| DEMASK2026.04 | 57.6 | 2.2 | |
| Diffusion (static)Model Scale=SDAR-4B-Chat, Decoding Strategy=Diffusion (static)2026.03 | 57.2 | 1.5 | |
| S2D2 (config-B)Model Scale=SDAR-4B-Chat, Decoding Strategy=S2D2 (config-B)2026.03 | 57 | 2.3 | |
| S2D2 (config-A)Model Scale=SDAR-4B-Chat, Decoding Strategy=S2D2 (config-A)2026.03 | 56 | 1.3 | |
| Diffusion (dynamic)Model Scale=SDAR-4B-Chat, Decoding Strategy=Diffusion (dynamic)2026.03 | 55.6 | 2.1 | |
| ARDecoding Method=Autoregressive, Block size (B)=4, Sub-block size (SB)=12026.03 | 51.8 | 1 | |
| DiffusionDecoding Method=Diffusion, Configuration=config-A, Block size (B)=322026.03 | 50.6 | 2.3 | |
| S2D2Decoding Method=S2D2, Configuration=config-A, Block size (B)=322026.03 | 48.8 | 2 | |
| ARModel Scale=SDAR-1.7B-Chat, Decoding Strategy=AR2026.03 | 48.4 | 1 | |
| DiffusionDecoding Method=Diffusion, Configuration=config-B, Block size (B)=322026.03 | 47.8 | 2.9 | |
| DiffusionDecoding Method=Diffusion, Configuration=config-C, Block size (B)=32, Sub-block size (SB)=322026.03 | 46.2 | 3.6 | |
| S2D2Decoding Method=S2D2, Configuration=config-C, Block size (B)=32, Sub-block size (SB)=322026.03 | 45.6 | 3.8 | |
| S2D2 (config-A)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=S2D2 (config-A)2026.03 | 45.4 | 1.5 | |
| S2D2Decoding Method=S2D2, Configuration=config-B, Block size (B)=322026.03 | 45 | 3.7 | |
| S2D2 (config-B)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=S2D2 (config-B)2026.03 | 44.4 | 2.9 | |
| EntropyTokens per step=22026.04 | 43.8 | 1.3 | |
| Top-1Tokens per step=22026.04 | 43.8 | 1.4 | |
| Token OrderTokens per step=22026.04 | 40.8 | 1.5 | |
| Diffusion (dynamic)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=Diffusion (dynamic)2026.03 | 21.6 | 1.3 | |
| Diffusion (static)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=Diffusion (static)2026.03 | 13.6 | 1.3 |