Speculative Decoding on Average Mixed Queries
7.15Match Rate (MAT)HedgeSpec
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HedgeSpecBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 7.15 | 90.41 | |
| HedgeSpecBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 6.37 | 69.44 | |
| HedgeSpecBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 6.21 | 40.41 | |
| EagleBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 5.69 | 74.34 | |
| UCBSpecBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 5.09 | 68.89 | |
| EXP3SpecBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 4.86 | 65.22 | |
| UCBSpecBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 4.58 | 52.72 | |
| UCBSpecBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 4.5 | 33.24 | |
| EXP3SpecBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 4.37 | 49.75 | |
| EXP3SpecBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 4.33 | 31.55 | |
| EagleBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 4.23 | 47.53 | |
| EagleBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 2.88 | 20.76 | |
| LLaMA-3.1-8B-ITBackbone=LLaMA-3.1-8B-IT, Inference Setting=Greedy2025.10 | 1 | 18.13 | |
| Qwen-3-8BBackbone=Qwen-3-8B, Inference Setting=Greedy2025.10 | 1 | 14.57 | |
| Qwen-3-32BBackbone=Qwen-3-32B, Inference Setting=Greedy2025.10 | 1 | 8.21 |