Code Generation on HumanEval base (Pass@1)
82.3Pass@1SDAR-8B
Evaluation results
| Method | Links | |
|---|---|---|
| SDAR-8BScale=8B, Tokens=55B, FLOPs=2640, Decoding Strategy=greedy static decoding, Block Size=42026.06 | 82.3 | |
| SDAR-4BScale=4B, Tokens=55B, FLOPs=1320, Decoding Strategy=greedy static decoding, Block Size=42026.06 | 76.8 | |
| Fast-dLLM-v2-7BScale=8B, Tokens=1B, FLOPs=42, Decoding Strategy=greedy static decoding, Block Size=42026.06 | 63.4 | |
| OPDLM-8BScale=8B, Tokens=0.066B, FLOPs=4.2, Decoding Strategy=greedy static decoding, Block Size=42026.06 | 59.8 | |
| SDAR-1.7BScale=1.7B, Tokens=55B, FLOPs (1e18)=561.02026.06 | 59.8 | |
| Dream-7BScale=8B, Tokens=580B, FLOPs=24360, Decoding Strategy=greedy static decoding, Block Size=42026.06 | 57.9 | |
| OPDLM-4BScale=4B, Tokens=0.076B, FLOPs=2.4, Decoding Strategy=greedy static decoding, Block Size=42026.06 | 56.1 | |
| Simple-dLLMScale=0.6B, Tokens=46B, FLOPs (1e18)=165.62026.06 | 45.7 | |
| Fast-dLLM-v2-1.5BScale=1.5B, Tokens=1B, FLOPs (1e18)=9.02026.06 | 43.9 | |
| LLaDA-8BScale=8B, Tokens=1500B, FLOPs=72000, Decoding Strategy=greedy static decoding, Block Size=42026.06 | 35.4 | |
| OPDLM-1.7BScale=1.7B, Tokens=0.072B, FLOPs (1e18)=0.982026.06 | 35.4 | |
| OPDLM-0.6BScale=0.6B, Tokens=0.048B, FLOPs (1e18)=0.232026.06 | 20.1 |