List your company
ResearchFormerly WizWand

Language Modeling on LM1B (test)

8.94Block EfficiencySpecTr-GBV
0.68242.82624.977.1138Oct 23, 2023Mar 19, 2024Aug 15, 2024Jan 10, 2025Jun 8, 2025Nov 3, 2025Apr 1, 2026
Updated 6mo ago

Evaluation results

MethodLinks
SpecTr-GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.048.941.9
SpecTrTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.048.931.82
GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.048.421.75
SDTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.048.171.71
SpecTr-GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.047.011.12
SpecTrTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.046.861.04
GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.046.421.03
SDTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.046.171.02
SpecTrK=8, L=82023.1042.13
SpecTrK=8, L=42023.103.12.08
SpeculativeK=1, L=82023.102.91.56
SpeculativeK=1, L=42023.102.41.67
Baseline2023.1011
ARTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.0411
ARTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.0411