Code Generation on HumanEval (Accuracy and Speedup)
79.3AccuracyDiffusion (static)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Diffusion (static)Model Scale=SDAR-8B-Chat, Decoding Strategy=Diffusion (static)2026.03 | 79.3 | 1.6 | |
| S2D2 (config-A)Model Scale=SDAR-8B-Chat, Decoding Strategy=S2D2 (config-A)2026.03 | 78.7 | 2.2 | |
| Diffusion (dynamic)Model Scale=SDAR-8B-Chat, Decoding Strategy=Diffusion (dynamic)2026.03 | 78 | 2.9 | |
| ARModel Scale=SDAR-4B-Chat, Decoding Strategy=AR2026.03 | 75.6 | 1 | |
| ARModel Scale=SDAR-8B-Chat, Decoding Strategy=AR2026.03 | 75.6 | 1 | |
| S2D2 (config-B)Model Scale=SDAR-8B-Chat, Decoding Strategy=S2D2 (config-B)2026.03 | 74.4 | 3.2 | |
| Diffusion (static)Model Scale=SDAR-4B-Chat, Decoding Strategy=Diffusion (static)2026.03 | 73.8 | 1.3 | |
| Diffusion (dynamic)Model Scale=SDAR-4B-Chat, Decoding Strategy=Diffusion (dynamic)2026.03 | 73.8 | 1.9 | |
| S2D2 (config-A)Model Scale=SDAR-4B-Chat, Decoding Strategy=S2D2 (config-A)2026.03 | 69.5 | 3.1 | |
| S2D2 (config-B)Model Scale=SDAR-4B-Chat, Decoding Strategy=S2D2 (config-B)2026.03 | 68.3 | 3.3 | |
| ARModel Scale=SDAR-1.7B-Chat, Decoding Strategy=AR2026.03 | 62.2 | 1 | |
| S2D2 (config-B)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=S2D2 (config-B)2026.03 | 52.4 | 4.1 | |
| Diffusion (static)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=Diffusion (static)2026.03 | 51.8 | 1.9 | |
| Diffusion (dynamic)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=Diffusion (dynamic)2026.03 | 51.8 | 3.4 | |
| S2D2 (config-A)Model Scale=SDAR-1.7B-Chat, Decoding Strategy=S2D2 (config-A)2026.03 | 48.8 | 2.3 |