Speculative Decoding on Law
132.69Throughput (tokens/s)EvoSpec
Evaluation Results
| Method | Links | |
|---|---|---|
| EvoSpecStatic Vocabulary Size=32k, Dynamic Buffer Size (Dyna)=2562026.04 | 132.69 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Buffer Size (Dyna)=5122026.04 | 132.51 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Buffer Size (Dyna)=10242026.04 | 132.23 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Buffer Size (Dyna)=10242026.04 | 128.41 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Buffer Size (Dyna)=5122026.04 | 124.33 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Buffer Size (Dyna)=2562026.04 | 119.54 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Vocabulary Size=1024, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 116.27 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Vocabulary Size=256, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 115.55 | |
| FR-SpecStatic Vocabulary Size=32k2026.04 | 114.61 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Vocabulary Size=256, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 114.27 | |
| EvoSpecStatic Vocabulary Size=32k, Dynamic Vocabulary Size=512, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 113.64 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Vocabulary Size=1024, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 113.29 | |
| EvoSpecStatic Vocabulary Size=16k, Dynamic Vocabulary Size=512, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 112.58 | |
| FR-SpecStatic Vocabulary Size=32k, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 112.54 | |
| Vanilla SD2026.04 | 112.2 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Buffer Size (Dyna)=10242026.04 | 110.12 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Buffer Size (Dyna)=5122026.04 | 108.57 | |
| Standard SDDraft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 107.12 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Vocabulary Size=512, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 106.29 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Buffer Size (Dyna)=2562026.04 | 105.19 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Vocabulary Size=1024, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 104.64 | |
| EvoSpecStatic Vocabulary Size=8k, Dynamic Vocabulary Size=256, Draft Model=Qwen3-1.7B, Target Model=Qwen3-4B, Architecture=Decoupled2026.04 | 98.8 |