List your company
ResearchFormerly WizWand

Speculative Decoding Efficiency Metrics on Average Performance

6.73SpeedupTAPS
1.24922.67214.0955.5179May 30, 2026Jun 3, 2026Jun 7, 2026Jun 12, 2026Jun 16, 2026Jun 20, 2026Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.056.737.92
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.056.147.91
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.055.917.89
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.055.727.83
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.055.138.89
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.054.916
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.054.866.03
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.054.285.75
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.054.245.69
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.054.035.48
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.053.875.74
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.053.58.9
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.053.488.88
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.052.994.19
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.052.974.2
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.052.658.95
HyperDFlashTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.062.533.36
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.052.396.48
HyperDFlashTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.062.252.97
MTPTemperature=0, Drafted steps per verification round=3, Decoding mode=Think-high mode2026.062.112.76
MTPTemperature=1, Drafted steps per verification round=3, Decoding mode=Think-high mode2026.061.962.56
EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.051.872.62
EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.051.813.05
EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.051.762.96
EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.051.722.95
EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.051.682.83
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.051.616.47
MTPTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.62.89
Vanilla DFlashTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.572.01
Vanilla DFlashTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.491.91
EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.051.482.09
MTPTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.462.65