Speculative Decoding on LM1B (test)
7.88BESpecTr-GBV
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecTr-GBVTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.88 | 1.28 | |
| SpecTrTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.81 | 1.17 | |
| GBVTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.52 | 1.16 | |
| SDTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.27 | 1.15 | |
| SpecTr-GBVTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 5.82 | 1.01 | |
| SpecTrTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 5.71 | 0.97 | |
| GBVTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 5.3 | 0.92 | |
| SDTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 5.04 | 0.91 | |
| ARTarget Model=CodeLlama-13B, Draft Model=CodeLlama-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 | |
| ARTarget Model=Vicuna-13B, Draft Model=Vicuna-7B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 |