Code Generation on HumanEval (Average Acceptance Length (τ), Speedup Ratio)
7.23Avg Acceptance Length (τ)PPOW
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PPOWModel=L31-8B, Temperature=0.02026.05 | 7.23 | 4.14 | |
| PPOWModel=Q3-8B, Temperature=0.02026.05 | 7.01 | 3.62 | |
| PPOWModel=L33-70B, Temperature=0.02026.05 | 6.96 | 4.82 | |
| PPOWModel=Q3-32B, Temperature=0.02026.05 | 6.91 | 4.16 | |
| EAGLE-3Model=L33-70B, Temperature=0.02026.05 | 6.78 | 4.8 | |
| GRIFFINModel=L31-8B, Temperature=0.02026.05 | 6.72 | 3.96 | |
| GRIFFINModel=L33-70B, Temperature=0.02026.05 | 6.69 | 4.78 | |
| EAGLE-3Model=Q3-8B, Temperature=0.02026.05 | 6.68 | 3.4 | |
| EAGLE-3Model=L31-8B, Temperature=0.02026.05 | 6.63 | 3.92 | |
| PPOWModel=L33-70B, Temperature=1.02026.05 | 6.57 | 4.62 | |
| EAGLE-3Model=Q3-32B, Temperature=0.02026.05 | 6.52 | 4.02 | |
| PPOWModel=L31-8B, Temperature=1.02026.05 | 6.33 | 3.37 | |
| GRIFFINModel=L33-70B, Temperature=1.02026.05 | 6.31 | 4.46 | |
| EAGLE-3Model=L33-70B, Temperature=1.02026.05 | 6.28 | 4.43 | |
| PPOWModel=Q3-32B, Temperature=1.02026.05 | 6.17 | 2.77 | |
| GRIFFINModel=L31-8B, Temperature=1.02026.05 | 6.01 | 3.29 | |
| PPOWModel=Q3-8B, Temperature=1.02026.05 | 5.96 | 2.33 | |
| EAGLE-3Model=L31-8B, Temperature=1.02026.05 | 5.94 | 3.25 | |
| EAGLE-3Model=Q3-8B, Temperature=1.02026.05 | 5.71 | 2.11 | |
| EAGLE-3Model=Q3-32B, Temperature=1.02026.05 | 5.53 | 2.59 |