Code Generation on MBPP (Accept Rate, pass@1)
62pass@1Autoregressive
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AutoregressiveTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 62 | — | |
| DIVERSEDTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 62 | 89.78 | |
| SDTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 61 | 72.94 | |
| SD (Lossy)Target/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 61 | 73.37 | |
| SpecCascadeTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 61 | 84.15 | |
| Static EnsembleTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 61 | 80.66 |