Speculative Decoding on HumanEval
6.99Speedup FactorTAPS
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 6.99 | — | 8.61 | — | — | — | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 6.85 | — | — | — | — | 9.51 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 6.79 | — | — | — | — | 9.4 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 6.47 | — | 8.44 | — | — | — | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 6.41 | — | 8.46 | — | — | — | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 6.36 | — | — | — | — | 8.55 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 6.23 | — | — | — | — | 8.67 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 6.22 | — | — | — | — | 8.35 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 6.05 | — | 8.52 | — | — | — | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.88 | — | — | — | — | 8.13 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.81 | — | — | — | — | 7.82 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 5.56 | — | 9.44 | — | — | — | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 5.5 | — | — | — | — | 7.83 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.4 | — | — | — | — | 7.26 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 5.4 | — | — | — | — | 7.74 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 5.39 | — | — | — | — | 7.43 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 5.25 | — | — | — | — | 7.2 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 5.21 | — | 6.59 | — | — | — | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 5.14 | — | 6.41 | — | — | — | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 5.02 | — | 6.59 | — | — | — | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.99 | — | — | — | — | 7.13 | |
| DFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 4.87 | — | — | — | — | 6.44 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.82 | — | — | — | — | 6.68 | |
| DFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 4.78 | — | — | — | — | 6.34 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 4.74 | — | 6.04 | — | — | — | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.63 | — | — | — | — | 6.63 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 4.46 | — | 6.55 | — | — | — | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.46 | — | — | — | — | 6.16 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 4.32 | — | 5.52 | — | — | — | |
| DFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.24 | — | — | — | — | 5.66 | |
| DFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 3.95 | — | — | — | — | 5.26 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 3.65 | — | 9.61 | — | — | — | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 3.58 | — | 9.65 | — | — | — | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 3.51 | — | 4.88 | — | — | — | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 3.48 | — | 4.9 | — | — | — | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 2.76 | — | 9.5 | — | — | — | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.74 | — | 7.28 | — | — | — | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 2.16 | — | — | — | — | 3.7 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.04 | — | 2.83 | — | — | — | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 2.03 | — | — | — | — | 3.55 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 2 | — | — | — | — | 3.47 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 1.91 | — | — | — | — | 3.39 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 1.89 | — | 3.17 | — | — | — | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 1.87 | — | — | — | — | 3.18 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 1.84 | — | 3.05 | — | — | — | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.81 | — | 7.26 | — | — | — | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 1.78 | — | — | — | — | 3.06 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 1.75 | — | 3.05 | — | — | — | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 1.75 | — | — | — | — | 3.05 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 1.74 | — | 3.01 | — | — | — | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.74 | — | 2.44 | — | — | — | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 1.7 | — | — | — | — | 2.96 | |
| CORAL2026.04 | — | — | — | 149.82 | — | — | |
| EAGLE-32026.04 | — | — | — | 136.95 | — | — | |
| EAGLE-3 KLTarget Model=LLaMA 3.1 8B Instruct, Temperature=02026.02 | — | — | — | — | 4.82 | — | |
| EAGLE-3 KLTarget Model=LLaMA 3.1 8B Instruct, Temperature=12026.02 | — | — | — | — | 4.31 | — | |
| EAGLE-3 KLTarget Model=LLaMA 3.3 70B Instruct, Temperature=02026.02 | — | — | — | — | 5.18 | — | |
| EAGLE-3 KLTarget Model=LLaMA 3.3 70B Instruct, Temperature=12026.02 | — | — | — | — | 4.86 | — | |
| EAGLE-3 KLTarget Model=GPT-OSS 20B, Temperature=02026.02 | — | — | — | — | 3.07 | — | |
| EAGLE-3 KLTarget Model=GPT-OSS 20B, Temperature=12026.02 | — | — | — | — | 2.89 | — | |
| EAGLE-3 KLTarget Model=GPT-OSS 120B, Temperature=02026.02 | — | — | — | — | 2.47 | — | |
| EAGLE-3 KLTarget Model=GPT-OSS 120B, Temperature=12026.02 | — | — | — | — | 2.27 | — | |
| EAGLE-3 KLTarget Model=Qwen3-235B-A22B-Instruct-2507, Temperature=02026.02 | — | — | — | — | 4.65 | — | |
| EAGLE-3 KLTarget Model=Qwen3-235B-A22B-Instruct-2507, Temperature=12026.02 | — | — | — | — | 4.09 | — | |
| EAGLE-3 Lλ LKTarget Model=LLaMA 3.1 8B Instruct, Temperature=02026.02 | — | — | — | — | 4.89 | — | |
| EAGLE-3 Lλ LKTarget Model=LLaMA 3.1 8B Instruct, Temperature=12026.02 | — | — | — | — | 4.52 | — | |
| EAGLE-3 Lλ LKTarget Model=LLaMA 3.3 70B Instruct, Temperature=02026.02 | — | — | — | — | 5.21 | — | |
| EAGLE-3 Lλ LKTarget Model=LLaMA 3.3 70B Instruct, Temperature=12026.02 | — | — | — | — | 5.08 | — | |
| EAGLE-3 Lλ LKTarget Model=GPT-OSS 20B, Temperature=02026.02 | — | — | — | — | 3.11 | — | |
| EAGLE-3 Lλ LKTarget Model=GPT-OSS 20B, Temperature=12026.02 | — | — | — | — | 3.01 | — | |
| EAGLE-3 Lλ LKTarget Model=GPT-OSS 120B, Temperature=02026.02 | — | — | — | — | 2.51 | — | |
| EAGLE-3 Lλ LKTarget Model=GPT-OSS 120B, Temperature=12026.02 | — | — | — | — | 2.44 | — | |
| EAGLE-3 Lλ LKTarget Model=Qwen3-235B-A22B-Instruct-2507, Temperature=02026.02 | — | — | — | — | 4.74 | — | |
| EAGLE-3 Lλ LKTarget Model=Qwen3-235B-A22B-Instruct-2507, Temperature=12026.02 | — | — | — | — | 4.42 | — | |
| EAGLE2model=Qwen2.5-Instruct-14B, Temperature=02025.03 | — | 2.84 | 3.1 | — | — | — | |
| EAGLE2model=LLaMA3-Instruct-8B, Temperature=02025.03 | — | 2.76 | 3.37 | — | — | — | |
| EAGLE2model=Qwen2.5-Instruct-14B, Temperature=12025.03 | — | 2.73 | 3.09 | — | — | — | |
| EAGLE2model=LLaMA3-Instruct-8B, Temperature=12025.03 | — | 2.57 | 3.38 | — | — | — | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Expansion Size=2562026.04 | — | — | — | 141.29 | — | — | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Expansion Size=5122026.04 | — | — | — | 141.32 | — | — | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Expansion Size=10242026.04 | — | — | — | 142.41 | — | — | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Expansion Size=2562026.04 | — | — | — | 156.5 | — | — | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Expansion Size=5122026.04 | — | — | — | 156.39 | — | — | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Expansion Size=10242026.04 | — | — | — | 156.31 | — | — | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Expansion Size=2562026.04 | — | — | — | 154.39 | — | — | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Expansion Size=5122026.04 | — | — | — | 153.23 | — | — | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Expansion Size=10242026.04 | — | — | — | 153.11 | — | — | |
| FR-SpecStatic Vocabulary Size=32k2026.04 | — | — | — | 137.06 | — | — | |
| MTP KLTarget Model=DeepSeek-V3-0324 (685B), Temperature=02026.02 | — | — | — | — | 4.74 | — | |
| MTP KLTarget Model=DeepSeek-V3-0324 (685B), Temperature=12026.02 | — | — | — | — | 4.39 | — | |
| MTP Lλ LKTarget Model=DeepSeek-V3-0324 (685B), Temperature=02026.02 | — | — | — | — | 4.77 | — | |
| MTP Lλ LKTarget Model=DeepSeek-V3-0324 (685B), Temperature=12026.02 | — | — | — | — | 4.64 | — | |
| MTP originalTarget Model=DeepSeek-V3-0324 (685B), Temperature=02026.02 | — | — | — | — | 3.28 | — | |
| MTP originalTarget Model=DeepSeek-V3-0324 (685B), Temperature=12026.02 | — | — | — | — | 3.17 | — | |
| PLDmodel=Qwen2.5-Instruct-14B, Temperature=02025.03 | — | 1.49 | 0.55 | — | — | — | |
| PLDmodel=LLaMA3-Instruct-8B, Temperature=02025.03 | — | 1.38 | 0.51 | — | — | — | |
| PLDmodel=Qwen2.5-Instruct-14B, Temperature=12025.03 | — | 1.53 | 0.58 | — | — | — | |
| PLDmodel=LLaMA3-Instruct-8B, Temperature=12025.03 | — | 1.32 | 0.45 | — | — | — | |
| RASD(PLD)model=Qwen2.5-Instruct-14B, Temperature=02025.03 | — | 2.98 | 3.44 | — | — | — | |
| RASD(PLD)model=LLaMA3-Instruct-8B, Temperature=02025.03 | — | 2.79 | 3.55 | — | — | — |