Speculative Decoding on Med
128.51Throughput (tokens/s)EvoSpec
Evaluation results
| Method | Links | |
|---|---|---|
| EvoSpecStatic Vocabulary Size=32k, Dynamic Buffer Size (Dyna)=2562026.04 | 128.51 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Buffer Size (Dyna)=5122026.04 | 128.23 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Buffer Size (Dyna)=10242026.04 | 127.8 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Buffer Size (Dyna)=10242026.04 | 122.88 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Buffer Size (Dyna)=5122026.04 | 118.46 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Buffer Size (Dyna)=2562026.04 | 116.21 | |
| FR-SpecStatic Vocabulary Size=32k2026.04 | 114.08 | |
| Vanilla SD2026.04 | 111.93 | |
| FR-SpecStatic Vocabulary Size=32k, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 111.35 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Vocabulary Size=256, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 110.85 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Vocabulary Size=512, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 110.47 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Vocabulary Size=1024, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 109.7 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Buffer Size (Dyna)=10242026.04 | 108.75 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Vocabulary Size=256, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 105.63 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Vocabulary Size=512, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 105.23 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Buffer Size (Dyna)=5122026.04 | 105.13 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Vocabulary Size=1024, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 102.72 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Buffer Size (Dyna)=2562026.04 | 101.47 | |
| Standard SDDraft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 98.58 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Vocabulary Size=1024, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 95.09 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Vocabulary Size=512, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 95.04 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Vocabulary Size=256, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 89.83 |