Speculative Decoding on GSM8K (test)
7.95BE ScoreSpecTr-GBV
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecTr-GBVTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.95 | 1.51 | |
| SpecTrTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.89 | 1.48 | |
| GBVTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.52 | 1.18 | |
| SDTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.33 | 1.18 | |
| SpecTr-GBVTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.74 | 1.67 | |
| SpecTrTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.61 | 1.56 | |
| GBVTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.13 | 1.15 | |
| SDTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 5.96 | 1.16 | |
| ARTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 | |
| ARTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 |