List your company
ResearchFormerly WizWand

Speculative Decoding on MATH 500

7.99SpeedupTreeFlash
0.98042.80024.626.4398May 30, 2026May 31, 2026Jun 1, 2026Jun 2, 2026
Updated 5mo ago

Evaluation results

MethodLinks
TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.067.9911.02
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.057.99.11
TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.067.910.99
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.057.419.16
DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.067.249.75
DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.067.159.56
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.056.599.4
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.056.559.41
TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.066.389.2
TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.066.348.84
TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.066.349.05
TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.066.38.7
DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.066.138.45
DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.066.128.42
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.056.097.18
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.056.087.25
DFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.065.967.93
DFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.065.967.88
DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.065.887.92
DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.065.737.65
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.055.6610.52
TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.065.127.31
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.055.096.67
DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.064.966.86
TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.064.887.02
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.054.846.4
DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.064.846.66
DFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.064.395.87
DFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.064.265.68
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.054.110.22
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.053.9510.4
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.053.765.34
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.053.465.37
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.053.1610.31
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.052.643.86
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.052.613.83
EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.062.233.81
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.052.26.1
EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.062.063.63
EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.061.953.32
EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.061.943.41
EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.061.913.41
EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.051.833.08
EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.051.813.02
EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.061.783.09
EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.051.752.99
EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.051.732.91
EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.061.722.94
EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.061.682.93
EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.051.542.24
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.051.446.09
EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.051.251.84