Instruction Following on Alpaca (test)
3.15SR ScoreCORAL
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CORALmodel=Llama2-7B-chat, temperature=02025.02 | 3.15 | 4.96 | — | |
| HASSmodel=Llama2-7B-chat, temperature=02025.02 | 3.11 | 4.87 | — | |
| EAGLE-2model=Llama2-7B-chat, temperature=02025.02 | 2.88 | 4.51 | — | |
| CORALmodel=Llama3-8B-Instruct, temperature=0, router=with router2025.02 | 2.74 | 4.49 | — | |
| CORALmodel=Qwen2.5-7B-Instruct, temperature=0, router=with router2025.02 | 2.6 | 4.11 | — | |
| CORALmodel=Llama3-8B-Instruct, temperature=02025.02 | 2.58 | 4.79 | — | |
| HASSmodel=Llama3-8B-Instruct, temperature=02025.02 | 2.56 | 4.77 | — | |
| EAGLE-2model=Llama3-8B-Instruct, temperature=02025.02 | 2.39 | 4.33 | — | |
| CORALmodel=Qwen2.5-7B-Instruct, temperature=02025.02 | 2.35 | 4.29 | — | |
| HASSmodel=Qwen2.5-7B-Instruct, temperature=02025.02 | 2.31 | 4.19 | — | |
| EAGLE-2model=Qwen2.5-7B-Instruct, temperature=02025.02 | 2.17 | 3.93 | — | |
| SpecTr-GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 1.6 | — | 7.58 | |
| SpecTrTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 1.52 | — | 7.42 | |
| GBVTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 1.41 | — | 6.89 | |
| SDTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 1.36 | — | 6.51 | |
| SpecTr-GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1.03 | — | 6.23 | |
| SDTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1.01 | — | 5.81 | |
| ARTarget Model=DeepSeek-33B, Draft Model=DeepSeek-1.3B, Draft length (L)=12, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | — | 1 | |
| ARTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | — | 1 | |
| SpecTrTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | — | 6.12 | |
| GBVTarget Model=DeepSeek-6.7B, Draft Model=DeepSeek-1.3B, Draft length (L)=8, Temperature (T)=0.4, Draft number (K)=32026.04 | 1 | — | 5.89 |