Mathematical Reasoning on GSM8K (Accept Rate, Accuracy)
76AccuracyAutoregressive
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AutoregressiveTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 76 | — | |
| SDTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 76 | 66.41 | |
| SD (Lossy)Target/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 76 | 67.42 | |
| SpecCascadeTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 76 | 68.01 | |
| DIVERSEDTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 76 | 89.14 | |
| Static EnsembleTarget/Draft model pair=Llama-3.1-8B/Llama-3.2-1B, Temperature=0.1, Draft length (N)=52026.04 | 75 | 72.36 |