Speculative Decoding on MBPP
6.75SpeedupTAPS
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 6.75 | 7.86 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 6.44 | 8.96 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 6.41 | 8.96 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 6.25 | 7.97 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 6.11 | 7.89 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.95 | 8.34 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 5.85 | 7.9 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 5.73 | 7.68 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 5.68 | 9 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 5.66 | 7.67 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.66 | 7.86 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.39 | 7.29 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.15 | 6.95 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 5.05 | 7.27 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 5.03 | 7.24 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 4.91 | 6.66 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 4.91 | 6.8 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 4.78 | 5.68 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 4.68 | 6.47 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.67 | 6.73 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 4.65 | 5.82 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.52 | 6.22 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 4.48 | 6.69 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.44 | 6.37 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 4.42 | 5.66 | |
| DFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 4.37 | 5.83 | |
| DFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 4.36 | 5.87 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.25 | 5.91 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 4.04 | 5.21 | |
| DFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 3.96 | 5.31 | |
| DFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 3.64 | 4.93 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 3.59 | 8.79 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 3.57 | 5.02 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 3.54 | 8.73 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 3.52 | 5.02 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.67 | 7.43 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 2.65 | 9.02 | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 1.96 | 3.35 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 1.94 | 3.38 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.87 | 2.62 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 1.87 | 3.32 | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 1.86 | 3.25 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.84 | 7.44 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 1.78 | 2.95 | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 1.74 | 2.97 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 1.69 | 2.82 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 1.69 | 2.89 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 1.69 | 2.94 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 1.64 | 2.74 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 1.64 | 2.88 | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 1.63 | 2.8 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.45 | 2.07 |