Speculative Decoding on Python
7.69MATHedgeSpec
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| HedgeSpecBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 7.69 | 99.58 | |
| HedgeSpecTarget Model=LLaMA-3.1-8B-IT, Inference Mode=Non-greedy2025.10 | 7.16 | 87.38 | |
| Vanilla EagleTarget Model=LLaMA-3.1-8B-IT, Inference Mode=Non-greedy2025.10 | 6.48 | 83.75 | |
| EagleBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 6.48 | 87.37 | |
| HedgeSpecBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 6.32 | 68.52 | |
| HedgeSpecTarget Model=Qwen-3-8B, Inference Mode=Non-greedy2025.10 | 5.82 | 62.63 | |
| HedgeSpecBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 5.82 | 38.44 | |
| UCBSpecBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 5.44 | 74.39 | |
| HedgeSpecTarget Model=Qwen-3-32B, Inference Mode=Non-greedy2025.10 | 5.31 | 36.86 | |
| UCBSpecTarget Model=LLaMA-3.1-8B-IT, Inference Mode=Non-greedy2025.10 | 5.17 | 64.67 | |
| EXP3SpecBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 5.16 | 69.29 | |
| EagleBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 4.96 | 55.84 | |
| EXP3SpecTarget Model=LLaMA-3.1-8B-IT, Inference Mode=Non-greedy2025.10 | 4.93 | 62.63 | |
| Vanilla EagleTarget Model=Qwen-3-8B, Inference Mode=Non-greedy2025.10 | 4.88 | 53.03 | |
| UCBSpecBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 4.75 | 54.8 | |
| EXP3SpecBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 4.54 | 52.77 | |
| UCBSpecTarget Model=Qwen-3-8B, Inference Mode=Non-greedy2025.10 | 4.45 | 50.84 | |
| EXP3SpecTarget Model=Qwen-3-8B, Inference Mode=Non-greedy2025.10 | 4.34 | 48 | |
| UCBSpecBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 4.34 | 31.42 | |
| EXP3SpecBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 4.22 | 30.33 | |
| UCBSpecTarget Model=Qwen-3-32B, Inference Mode=Non-greedy2025.10 | 4.12 | 30.36 | |
| EXP3SpecTarget Model=Qwen-3-32B, Inference Mode=Non-greedy2025.10 | 4.01 | 28.27 | |
| Vanilla EagleTarget Model=Qwen-3-32B, Inference Mode=Non-greedy2025.10 | 3.03 | 22.91 | |
| EagleBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 3.02 | 21.98 | |
| LLaMA-3.1-8B-ITTarget Model=LLaMA-3.1-8B-IT, Inference Mode=Non-greedy2025.10 | 1 | 17.77 | |
| Qwen-3-8BTarget Model=Qwen-3-8B, Inference Mode=Non-greedy2025.10 | 1 | 15.33 | |
| Qwen-3-32BTarget Model=Qwen-3-32B, Inference Mode=Non-greedy2025.10 | 1 | 8.52 | |
| LLaMA-3.1-8B-ITBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 1 | 17.03 | |
| Qwen-3-8BBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 1 | 14.55 | |
| Qwen-3-32BBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 1 | 8.13 |