Code Generation on HumanEval (Speedup, tau)
5.89SpeedupDOMINO
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DOMINOTemperature=0, Model=Qwen3-8B, Draft size=162026.05 | 5.89 | 7.39 | |
| DOMINOTemperature=0, Model=Qwen3-4B, Draft size=162026.05 | 5.66 | 7.02 | |
| DFlashTemperature=0, Model=Qwen3-4B, Draft size=162026.05 | 5.23 | 6.63 | |
| DFlashTemperature=0, Model=Qwen3-8B, Draft size=162026.05 | 5.21 | 6.52 | |
| DOMINOTemperature=1, Model=Qwen3-4B, Draft size=162026.05 | 5.02 | 6.3 | |
| DOMINOTemperature=1, Model=Qwen3-8B, Draft size=162026.05 | 4.75 | 5.98 | |
| DFlashTemperature=1, Model=Qwen3-4B, Draft size=162026.05 | 4.73 | 5.96 | |
| DFlashTemperature=1, Model=Qwen3-8B, Draft size=162026.05 | 4.34 | 5.44 | |
| DARTTemperature=0, Model=Qwen3-8B, Draft size=602026.05 | 2.52 | 2.95 | |
| EAGLE-3Temperature=0, Model=Qwen3-8B, Draft size=602026.05 | 2.5 | 3.74 | |
| DARTTemperature=0, Model=Qwen3-4B, Draft size=602026.05 | 2.48 | 2.99 | |
| DARTTemperature=1, Model=Qwen3-4B, Draft size=602026.05 | 2.48 | 2.99 | |
| DARTTemperature=1, Model=Qwen3-8B, Draft size=602026.05 | 2.45 | 2.95 | |
| EAGLE-3Temperature=0, Model=Qwen3-4B, Draft size=602026.05 | 2.36 | 3.53 | |
| EAGLE-3Temperature=1, Model=Qwen3-8B, Draft size=602026.05 | 2.32 | 3.58 | |
| EAGLE-3Temperature=0, Model=Qwen3-8B, Draft size=162026.05 | 2.17 | 3.21 | |
| EAGLE-3Temperature=0, Model=Qwen3-4B, Draft size=162026.05 | 2.09 | 3.09 | |
| EAGLE-3Temperature=1, Model=Qwen3-4B, Draft size=602026.05 | 2.09 | 3.09 | |
| EAGLE-3Temperature=1, Model=Qwen3-8B, Draft size=162026.05 | 2.05 | 3.09 | |
| EAGLE-3Temperature=1, Model=Qwen3-4B, Draft size=162026.05 | 2.02 | 3.04 |