Code Generation on HumanEval (Speedup and Generation Length)
7.12SpeedupJetFlow
Evaluation results
| Method | Links | ||
|---|---|---|---|
| JetFlowBudget=256, Temperature=02026.06 | 7.12 | 7.78 | |
| JetFlowBudget=128, Temperature=02026.06 | 6.66 | 7.28 | |
| DFlash-TBudget=256, Temperature=02026.06 | 6.31 | 6.96 | |
| JetFlowBudget=256, Temperature=12026.06 | 6.19 | 6.85 | |
| DFlash-TBudget=128, Temperature=02026.06 | 5.93 | 6.52 | |
| JetFlowBudget=128, Temperature=12026.06 | 5.82 | 6.39 | |
| JetFlowBudget=64, Temperature=02026.06 | 5.53 | 6.06 | |
| DFlash-TBudget=256, Temperature=12026.06 | 5.43 | 6.03 | |
| DFlash-TBudget=128, Temperature=12026.06 | 5.29 | 5.76 | |
| DFlash-TBudget=64, Temperature=02026.06 | 5.08 | 5.57 | |
| JetFlowBudget=64, Temperature=12026.06 | 5.02 | 5.52 | |
| DFlash-TBudget=64, Temperature=12026.06 | 4.65 | 5.09 | |
| EAGLE-3Budget=64, Temperature=02026.06 | 2.49 | 4.26 | |
| EAGLE-3Budget=64, Temperature=12026.06 | 2.35 | 4.13 |