List your company
ResearchFormerly WizWand

Instruction Following on Alpaca (test)

3.15SR ScoreCORAL
0.9141.49452.0752.6555Feb 24, 2025May 1, 2025Jul 7, 2025Sep 12, 2025Nov 18, 2025Jan 24, 2026Apr 1, 2026
Updated 6mo ago

Evaluation results

MethodLinks
CORALmodel=Llama2-7B-chat, temperature=02025.023.154.96—
HASSmodel=Llama2-7B-chat, temperature=02025.023.114.87—
EAGLE-2model=Llama2-7B-chat, temperature=02025.022.884.51—
CORALmodel=Llama3-8B-Instruct, temperature=0, router=with router2025.022.744.49—
CORALmodel=Qwen2.5-7B-Instruct, temperature=0, router=with router2025.022.64.11—
CORALmodel=Llama3-8B-Instruct, temperature=02025.022.584.79—
HASSmodel=Llama3-8B-Instruct, temperature=02025.022.564.77—
EAGLE-2model=Llama3-8B-Instruct, temperature=02025.022.394.33—
CORALmodel=Qwen2.5-7B-Instruct, temperature=02025.022.354.29—
HASSmodel=Qwen2.5-7B-Instruct, temperature=02025.022.314.19—
EAGLE-2model=Qwen2.5-7B-Instruct, temperature=02025.022.173.93—
SpecTr-GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.041.6—7.58
SpecTrTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.041.52—7.42
GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.041.41—6.89
SDTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.041.36—6.51
SpecTr-GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.041.03—6.23
SDTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.041.01—5.81
ARTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.041—1
ARTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.041—1
SpecTrTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.041—6.12
GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.041—5.89