List your company
ResearchFormerly WizWand

Speculative Decoding on gsm8k

9.43Average Generation Length (τ)TreeFlash
2.80524.52516.2457.9649Feb 27, 2026Mar 14, 2026Mar 30, 2026Apr 15, 2026May 1, 2026May 17, 2026Jun 2, 2026
Updated 4mo ago

Evaluation results

MethodLinks
TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.069.43—6.73
TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.069.39—6.7
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.059.27—3.97
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.059.26—5.87
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.059.25—2.84
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.059.14—3.57
TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.068.72—6.22
TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.068.69—6.21
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.058.26—6.98
DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.068.25—6.15
DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.068.24—6.07
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.058.23—6.62
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.058.17—6.55
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.058.11—6.57
DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.067.74—5.71
TreeFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.067.71—5.33
DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.067.67—5.67
TreeFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.067.62—5.29
TreeFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.067.15—4.98
DDTreeTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.067.14—5.15
DDTreeTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.067.13—5.13
TreeFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.067.03—4.85
DDTreeTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.066.66—4.78
DDTreeTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.066.62—4.77
DDTreeHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.056.53—1.66
DDTreeHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.056.53—2.41
DFlashTarget model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.066.33—4.71
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.056.29—5.15
DFlashTarget model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.066.26—4.63
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.056.12—5.15
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.056—4.71
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.055.98—4.67
TAPSHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.055.82—4.38
TAPSHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.055.78—4
MTP Lλ LKTarget Model=DeepSeek-V3-0324 (685B), Temperature=02026.025.72——
MTP KLTarget Model=DeepSeek-V3-0324 (685B), Temperature=02026.025.67——
DFlashTarget model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.065.65—4.19
DFlashTarget model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.065.65—4.21
HASSModel Variant=Mixed 70k+70k, Temperature=02026.035.53——
MTP Lλ LKTarget Model=DeepSeek-V3-0324 (685B), Temperature=12026.025.51——
Griffin+GSBackbone=Meta-Llama-3-8B-Instruct, Decoding Strategy=greedy decoding2026.035.47—3.19
Griffin+LTDBackbone=Meta-Llama-3-8B-Instruct, Decoding Strategy=greedy decoding2026.035.45—3.13
HASSModel Variant=Merged Trees, Temperature=02026.035.42——
EAGLE-2Model Variant=Merged Trees, Temperature=02026.035.32——
MTP KLTarget Model=DeepSeek-V3-0324 (685B), Temperature=12026.025.23——
HASSModel Variant=Merged Trees, Temperature=12026.035.21——
EAGLE-3 Lλ LKTarget Model=LLaMA 3.3 70B Instruct, Temperature=02026.025.21——
GriffinBackbone=Meta-Llama-3-8B-Instruct, Decoding Strategy=greedy decoding2026.035.16—3.18
EAGLE-3 KLTarget Model=LLaMA 3.3 70B Instruct, Temperature=02026.025.16——
EAGLE-2Model Variant=MathInstruct, Temperature=02026.035.04——
HASSModel Variant=MathInstruct, Temperature=02026.035.02——
HASSModel Variant=Confidence Routed, Temperature=02026.035.01——
EAGLE-2Model Variant=Merged Trees, Temperature=12026.035.01——
EAGLE-3 Lλ LKTarget Model=LLaMA 3.3 70B Instruct, Temperature=12026.025.01——
EAGLE-2Model Variant=Confidence Routed, Temperature=02026.034.91——
EAGLE-3 KLTarget Model=LLaMA 3.3 70B Instruct, Temperature=12026.024.89——
EAGLE-3 Lλ LKTarget Model=Qwen3-235B-A22B-Instruct-2507, Temperature=02026.024.87——
HASSModel Variant=Mixed 35k+35k, Temperature=02026.034.77——
EAGLE-3 KLTarget Model=Qwen3-235B-A22B-Instruct-2507, Temperature=02026.024.76——
HASSModel Variant=MathInstruct, Temperature=12026.034.75——
HASSModel Variant=Confidence Routed, Temperature=12026.034.72——
EAGLE-2Model Variant=MathInstruct, Temperature=12026.034.71——
EAGLE-2Model Variant=Mixed 70k+70k, Temperature=02026.034.68——
HASSModel Variant=Mixed 35k+35k, Temperature=12026.034.66——
EAGLE-2Model Variant=Confidence Routed, Temperature=12026.034.65——
EAGLE-3 Lλ LKTarget Model=Qwen3-235B-A22B-Instruct-2507, Temperature=12026.024.65——
EAGLE-3 Lλ LKTarget Model=LLaMA 3.1 8B Instruct, Temperature=02026.024.57——
EAGLE-3 KLTarget Model=LLaMA 3.1 8B Instruct, Temperature=02026.024.5——
DFlashHardware=NVIDIA A800-SXM4-80GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.054.27—3.09
EAGLE-3 KLTarget Model=Qwen3-235B-A22B-Instruct-2507, Temperature=12026.024.27——
DFlashHardware=NVIDIA A40-48GB GPU, Target Model=LLaMA-3.1-8B-Instruct2026.054.25—3.06
HASSModel Variant=Mixed 70k+70k, Temperature=12026.034.16——
EAGLE-2Model Variant=Mixed 35k+35k, Temperature=02026.034.12——
HASSModel Variant=ShareGPT, Temperature=02026.034.09——
EAGLE-2Model Variant=Mixed 35k+35k, Temperature=12026.034.08——
HASSModel Variant=ShareGPT, Temperature=12026.034.03——
EAGLE-3 Lλ LKTarget Model=LLaMA 3.1 8B Instruct, Temperature=12026.024.02——
EAGLE-3 KLTarget Model=GPT-OSS 20B, Temperature=02026.024.01——
EAGLE-3 Lλ LKTarget Model=GPT-OSS 20B, Temperature=02026.024——
EAGLE-3 KLTarget Model=LLaMA 3.1 8B Instruct, Temperature=12026.023.88——
EAGLE-2Model Variant=Mixed 70k+70k, Temperature=12026.033.76——
EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=642026.063.75—2.17
EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=642026.063.73—2.14
EAGLE-2Model Variant=ShareGPT, Temperature=02026.033.72——
EAGLE-2Model Variant=ShareGPT, Temperature=12026.033.72——
EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=642026.063.66—2.06
EAGLE-3 Lλ LKTarget Model=GPT-OSS 20B, Temperature=12026.023.65——
EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=642026.063.63—2.06
EAGLE-3 KLTarget Model=GPT-OSS 20B, Temperature=12026.023.51——
MTP originalTarget Model=DeepSeek-V3-0324 (685B), Temperature=02026.023.41——
EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-4B2026.053.3—1.99
MTP originalTarget Model=DeepSeek-V3-0324 (685B), Temperature=12026.023.27——
EAGLE-3Hardware=NVIDIA A40-48GB GPU, Target Model=Qwen3-8B2026.053.23—1.94
EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-4B2026.053.22—1.89
EAGLE3Target model=Qwen3 8B, Temperature (T)=0, Draft budget (B)=162026.063.21—1.88
EAGLE3Target model=Qwen3 4B, Temperature (T)=0, Draft budget (B)=162026.063.18—1.84
EAGLE3Target model=Qwen3 4B, Temperature (T)=1, Draft budget (B)=162026.063.15—1.79
EAGLE3Target model=Qwen3 8B, Temperature (T)=1, Draft budget (B)=162026.063.13—1.82
EAGLE-3Hardware=NVIDIA A800-SXM4-80GB GPU, Target Model=Qwen3-8B2026.053.12—1.87
EAGLE-3 Lλ LKTarget Model=GPT-OSS 120B, Temperature=02026.023.06——