Code Generation on MBPP (Speedup, tau)
4.78SpeedupDFlash
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DFlashModel=Q3-4B, Temperature=0, Diffusion block size=162026.02 | 4.78 | 6.09 | |
| DFlashModel=Q3-8B, Temperature=0, Diffusion block size=162026.02 | 4.65 | 5.95 | |
| DFlashModel=Q3-4B, Temperature=1, Diffusion block size=162026.02 | 4.42 | 5.66 | |
| DFlashModel=Q3-8B, Temperature=1, Diffusion block size=162026.02 | 4.04 | 5.21 | |
| EAGLE-3Model=Q3-4B, Temperature=0, Draft tree size=602026.02 | 2.02 | 3.38 | |
| EAGLE-3Model=Q3-8B, Temperature=0, Draft tree size=602026.02 | 1.93 | 3.25 | |
| EAGLE-3Model=Q3-4B, Temperature=1, Draft tree size=602026.02 | 1.92 | 3.33 | |
| EAGLE-3Model=Q3-8B, Temperature=1, Draft tree size=602026.02 | 1.85 | 3.16 | |
| EAGLE-3Model=Q3-4B, Temperature=0, Draft tree size=162026.02 | 1.78 | 2.95 | |
| EAGLE-3Model=Q3-8B, Temperature=0, Draft tree size=162026.02 | 1.69 | 2.82 | |
| EAGLE-3Model=Q3-4B, Temperature=1, Draft tree size=162026.02 | 1.69 | 2.89 | |
| EAGLE-3Model=Q3-8B, Temperature=1, Draft tree size=162026.02 | 1.64 | 2.74 |