Speculative Decoding on GPQA Diamond
91.17ThroughputSTAND
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| STANDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=162025.06 | 91.17 | 3.9 | 2.91 | |
| STANDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=82025.06 | 87.02 | 3.7 | 2.78 | |
| STANDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=42025.06 | 83.47 | 3.48 | 2.66 | |
| RecycleBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=82025.06 | 71.55 | 2.73 | 2.28 | |
| RecycleBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=42025.06 | 71.51 | 2.73 | 2.28 | |
| RecycleBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=162025.06 | 71.23 | 2.73 | 2.27 | |
| SAM+RecycleBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=42025.06 | 70.43 | 2.73 | 2.25 | |
| SAM+RecycleBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=82025.06 | 69.98 | 2.74 | 2.23 | |
| SAM+RecycleBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=162025.06 | 69.85 | 2.71 | 2.23 | |
| ANPDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=162025.06 | 60.25 | 2.11 | 1.92 | |
| ANPDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=82025.06 | 58.97 | 2.03 | 1.88 | |
| SAMBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=162025.06 | 57.53 | 2.03 | 1.84 | |
| ANPDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=42025.06 | 57.39 | 1.97 | 1.83 | |
| SAMBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=82025.06 | 55.47 | 1.96 | 1.77 | |
| PLDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=162025.06 | 53.47 | 1.96 | 1.71 | |
| SAMBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=42025.06 | 53.21 | 1.87 | 1.7 | |
| PLDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=82025.06 | 53.04 | 1.89 | 1.69 | |
| PLDBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=42025.06 | 50.49 | 1.81 | 1.61 | |
| STANDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=162025.06 | 49.11 | 3.86 | 2.7 | |
| STANDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=82025.06 | 46.81 | 3.63 | 2.58 | |
| STANDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=42025.06 | 43.71 | 3.42 | 2.41 | |
| RecycleBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=42025.06 | 38.99 | 2.73 | 2.15 | |
| SAM+RecycleBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=162025.06 | 38.99 | 2.71 | 2.15 | |
| RecycleBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=82025.06 | 38.89 | 2.73 | 2.14 | |
| RecycleBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=162025.06 | 38.81 | 2.74 | 2.14 | |
| SAM+RecycleBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=82025.06 | 38.57 | 2.71 | 2.12 | |
| SAM+RecycleBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=42025.06 | 38.24 | 2.71 | 2.11 | |
| Eagle-2Base Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=42025.06 | 31.69 | 1.99 | 1.01 | |
| Eagle-2Base Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=82025.06 | 31.69 | 1.99 | 1.01 | |
| Eagle-2Base Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=162025.06 | 31.69 | 1.99 | 1.01 | |
| PlainBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=42025.06 | 31.34 | — | — | |
| PlainBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=82025.06 | 31.34 | — | — | |
| PlainBase Model=DeepSeek-R1-Distill-Qwen-7B, Trajectories=162025.06 | 31.34 | — | — | |
| ANPDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=162025.06 | 30.62 | 2.13 | 1.69 | |
| SAMBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=162025.06 | 30.59 | 2.06 | 1.68 | |
| ANPDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=82025.06 | 29.51 | 2.04 | 1.63 | |
| SAMBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=82025.06 | 29.39 | 1.95 | 1.62 | |
| PLDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=162025.06 | 28.43 | 2 | 1.57 | |
| ANPDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=42025.06 | 28.21 | 1.97 | 1.55 | |
| SAMBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=42025.06 | 28.03 | 1.85 | 1.54 | |
| PLDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=82025.06 | 27.36 | 1.91 | 1.51 | |
| PLDBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=42025.06 | 26.6 | 1.82 | 1.46 | |
| Eagle-2Base Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=42025.06 | 24.86 | 2.44 | 1.37 | |
| Eagle-2Base Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=82025.06 | 24.86 | 2.44 | 1.37 | |
| Eagle-2Base Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=162025.06 | 24.86 | 2.44 | 1.37 | |
| PlainBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=42025.06 | 18.16 | — | — | |
| PlainBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=82025.06 | 18.16 | — | — | |
| PlainBase Model=DeepSeek-R1-Distill-Qwen-14B, Trajectories=162025.06 | 18.16 | — | — |