Speculative Decoding on MATH 500
7.99SpeedupTreeFlash
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 7.99 | 11.02 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 7.9 | 9.11 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 7.9 | 10.99 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 7.41 | 9.16 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 7.24 | 9.75 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 7.15 | 9.56 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 6.59 | 9.4 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 6.55 | 9.41 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 6.38 | 9.2 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 6.34 | 8.84 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 6.34 | 9.05 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 6.3 | 8.7 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 6.13 | 8.45 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 6.12 | 8.42 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 6.09 | 7.18 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 6.08 | 7.25 | |
| DFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 5.96 | 7.93 | |
| DFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 5.96 | 7.88 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.88 | 7.92 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 5.73 | 7.65 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 5.66 | 10.52 | |
| TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 5.12 | 7.31 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 5.09 | 6.67 | |
| DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.96 | 6.86 | |
| TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.88 | 7.02 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 4.84 | 6.4 | |
| DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.84 | 6.66 | |
| DFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.39 | 5.87 | |
| DFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 4.26 | 5.68 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 4.1 | 10.22 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 3.95 | 10.4 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 3.76 | 5.34 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 3.46 | 5.37 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 3.16 | 10.31 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.64 | 3.86 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.61 | 3.83 | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.06 | 2.23 | 3.81 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.2 | 6.1 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.06 | 2.06 | 3.63 | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.06 | 1.95 | 3.32 | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.06 | 1.94 | 3.41 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.06 | 1.91 | 3.41 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 1.83 | 3.08 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 1.81 | 3.02 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.06 | 1.78 | 3.09 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 1.75 | 2.99 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 1.73 | 2.91 | |
| EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.06 | 1.72 | 2.94 | |
| EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.06 | 1.68 | 2.93 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.54 | 2.24 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.44 | 6.09 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.25 | 1.84 |