Speculative Decoding Efficiency Metrics on Average Performance
6.73SpeedupTAPS
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 6.73 | 7.92 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 6.14 | 7.91 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 5.91 | 7.89 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 5.72 | 7.83 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 5.13 | 8.89 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 4.91 | 6 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 4.86 | 6.03 | |
| TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 4.28 | 5.75 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 4.24 | 5.69 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 4.03 | 5.48 | |
| TAPSHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 3.87 | 5.74 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 3.5 | 8.9 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 3.48 | 8.88 | |
| DFlashHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.99 | 4.19 | |
| DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.97 | 4.2 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 2.65 | 8.95 | |
| HyperDFlashTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.06 | 2.53 | 3.36 | |
| DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 2.39 | 6.48 | |
| HyperDFlashTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.06 | 2.25 | 2.97 | |
| MTPTemperature=0, Drafted steps per verification round=3, Decoding mode=Think-high mode2026.06 | 2.11 | 2.76 | |
| MTPTemperature=1, Drafted steps per verification round=3, Decoding mode=Think-high mode2026.06 | 1.96 | 2.56 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.87 | 2.62 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.05 | 1.81 | 3.05 | |
| EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.05 | 1.76 | 2.96 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.05 | 1.72 | 2.95 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.05 | 1.68 | 2.83 | |
| DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.61 | 6.47 | |
| MTPTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.06 | 1.6 | 2.89 | |
| Vanilla DFlashTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.06 | 1.57 | 2.01 | |
| Vanilla DFlashTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.06 | 1.49 | 1.91 | |
| EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.05 | 1.48 | 2.09 | |
| MTPTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.06 | 1.46 | 2.65 |