List your company
ResearchFormerly WizWand

Speculative Decoding on MGSM (test)

7.39BESpecTr-GBV
0.74442.46974.1955.9203Apr 1, 2026
Updated 6mo ago

Evaluation results

MethodLinks
SpecTr-GBVTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.047.391.51
SpecTrTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.047.361.39
GBVTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.046.891.26
SDTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.046.831.28
SpecTr-GBVTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.046.651.3
SpecTrTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.046.41.16
GBVTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.046.111.09
SDTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.045.941.09
ARTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.0411
ARTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.0411