Speculative Decoding on MGSM (test)
7.39BESpecTr-GBV
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SpecTr-GBVTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.39 | 1.51 | |
| SpecTrTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.36 | 1.39 | |
| GBVTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.89 | 1.26 | |
| SDTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.83 | 1.28 | |
| SpecTr-GBVTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.65 | 1.3 | |
| SpecTrTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.4 | 1.16 | |
| GBVTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.11 | 1.09 | |
| SDTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 5.94 | 1.09 | |
| ARTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 | |
| ARTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 |