Mathematical Reasoning on GSM8K (AVGLEN & SPEEDUP)
6.6Average LengthDraft-OPD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 6.6 | 6.22 | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 6.57 | 6.49 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 6.13 | 5.84 | |
| DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 6.04 | 5.36 | |
| DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 6.04 | 5.69 | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 6.02 | 6.01 | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 5.99 | 4.99 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 5.96 | 5.31 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 5.84 | 4.58 | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 5.73 | 5.36 | |
| DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.73 | 5.04 | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.69 | 4.61 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.66 | 4.16 | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 5.64 | 4.58 | |
| DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.62 | 5.26 | |
| DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 5.51 | 4.51 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 5.33 | 4.41 | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 5.33 | 4.13 | |
| DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 5.19 | 4.67 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 5.13 | 4.73 | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 5.05 | 4.77 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 5.03 | 3.9 | |
| DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.77 | 4.21 | |
| DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.66 | 4.23 | |
| CATS w/ EAGLEDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 4.5345 | 4.55 | |
| CATSDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.9652 | 4.12 | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.7629 | 2.85 | |
| CATS w/ EAGLEDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.6869 | 3.7 | |
| KangarooDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.0513 | 2.34 | |
| CATSDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.0198 | 3.14 | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.0059 | 2.26 | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.9515 | 2.23 | |
| Opt-HydraTarget model=meta-llama / Llama-2-7B-Chat2026.03 | 2.94 | 1.9 | |
| LookaheadDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 2.701 | 2.72 | |
| Opt-HydraTarget model=lmsys / Vicuna-7B-v1.32026.03 | 2.69 | 1.26 | |
| MedusaDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.6837 | 2.7 | |
| OSD-HydraTarget model=lmsys / Vicuna-7B-v1.32026.03 | 2.56 | 1.19 | |
| OSD-HydraTarget model=meta-llama / Llama-2-7B-Chat2026.03 | 2.55 | 1.67 | |
| MedusaDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.4966 | 2.51 | |
| KangarooDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.4856 | 1.92 | |
| Ens-EAGLE-3Target model=meta-llama / Llama-2-7B-Chat2026.03 | 2.33 | 1.27 | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.3213 | 1.76 | |
| OSD-EAGLE-3Target model=meta-llama / Llama-2-7B-Chat2026.03 | 2.25 | 1.23 | |
| Ens-EAGLE-3Target model=lmsys / Vicuna-7B-v1.32026.03 | 2.16 | 1.26 | |
| HydraTarget model=lmsys / Vicuna-7B-v1.32026.03 | 2.14 | 1 | |
| OSD-EAGLE-3Target model=lmsys / Vicuna-7B-v1.32026.03 | 2.07 | 1.2 | |
| Ens-EAGLETarget model=lmsys / Vicuna-7B-v1.32026.03 | 2.01 | 1.41 | |
| OSD-EAGLETarget model=lmsys / Vicuna-7B-v1.32026.03 | 1.92 | 1.28 | |
| EAGLE-3Target model=meta-llama / Llama-2-7B-Chat2026.03 | 1.82 | 1 | |
| EAGLE-3Target model=lmsys / Vicuna-7B-v1.32026.03 | 1.78 | 1 | |
| Ens-EAGLETarget model=meta-llama / Llama-2-7B-Chat2026.03 | 1.58 | 1.33 | |
| OSDTarget model=lmsys / Vicuna-7B-v1.32026.03 | 1.52 | 1.23 | |
| HydraTarget model=meta-llama / Llama-2-7B-Chat2026.03 | 1.52 | 1 | |
| RESTDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.5192 | 1.55 | |
| RESTDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.5045 | 1.54 | |
| EAGLETarget model=lmsys / Vicuna-7B-v1.32026.03 | 1.48 | 1 | |
| OSD-EAGLETarget model=meta-llama / Llama-2-7B-Chat2026.03 | 1.45 | 1.18 | |
| OSDTarget model=meta-llama / Llama-2-7B-Chat2026.03 | 1.4 | 1.06 | |
| Vanilla SDTarget model=lmsys / Vicuna-7B-v1.32026.03 | 1.25 | 1 | |
| Vanilla SDTarget model=meta-llama / Llama-2-7B-Chat2026.03 | 1.25 | 1 | |
| EAGLETarget model=meta-llama / Llama-2-7B-Chat2026.03 | 1.19 | 1 |