Language Modeling on LM1B (test)
8.94Block EfficiencySpecTr-GBV
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SpecTr-GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 8.94 | 1.9 | |
| SpecTrTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 8.93 | 1.82 | |
| GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 8.42 | 1.75 | |
| SDTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 8.17 | 1.71 | |
| SpecTr-GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 7.01 | 1.12 | |
| SpecTrTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.86 | 1.04 | |
| GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.42 | 1.03 | |
| SDTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 6.17 | 1.02 | |
| SpecTrK=8, L=82023.10 | 4 | 2.13 | |
| SpecTrK=8, L=42023.10 | 3.1 | 2.08 | |
| SpeculativeK=1, L=82023.10 | 2.9 | 1.56 | |
| SpeculativeK=1, L=42023.10 | 2.4 | 1.67 | |
| Baseline2023.10 | 1 | 1 | |
| ARTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 | |
| ARTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | 1 |