Speculative Decoding on SpecBench
900.7AVG SRMicroSpec
Evaluation results
| Method | Links | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MicroSpecModel=Llama-3.2-1B-Instruct2026.04 | 900.7 | — | — | — | — | 928.5 | — | — | — | — | — | — | — | 796.2 | — | — | — | 822 | — | 830.4 | — | — | — | — | 3.11 | — | — | — | — | 850.6 | 1,008.2 | 1,069.3 | |
| DynaSpecModel=Llama-3.2-1B-Instruct2026.04 | 799.2 | — | — | — | — | 951.9 | — | — | — | — | — | — | — | 667.7 | — | — | — | 717 | — | 721.2 | — | — | — | — | — | — | — | — | — | 763.9 | 865.7 | 907 | |
| FR-SpecModel=Llama-3.2-1B-Instruct2026.04 | 778.8 | — | — | — | — | 835.9 | — | — | — | — | — | — | — | 672.8 | — | — | — | 718.5 | — | 715.2 | — | — | — | — | 2.7 | — | — | — | — | 751.2 | 844.7 | 913.5 | |
| CORALModel=Llama-3.2-1B-Instruct2026.04 | 749.4 | — | — | — | — | 843.7 | — | — | — | — | — | — | — | 644.8 | — | — | — | 683 | — | 693.1 | — | — | — | — | — | — | — | — | — | 696.4 | 823.9 | 861.1 | |
| EAGLEModel=Llama-3.2-1B-Instruct2026.04 | 700.9 | — | — | — | — | 753.4 | — | — | — | — | — | — | — | 608.3 | — | — | — | 644.4 | — | 653.8 | — | — | — | — | 3.16 | — | — | — | — | 657 | 777.3 | 812.3 | |
| ARModel=Llama-3.2-1B-Instruct2026.04 | 665.8 | — | — | — | — | 671.3 | — | — | — | — | — | — | — | 661.7 | — | — | — | 672.2 | — | 624.7 | — | — | — | — | 1 | — | — | — | — | 675.6 | 677.2 | 677.9 | |
| MicroSpecModel=Llama-3-8B-Instruct2026.04 | 392.7 | — | — | — | — | 381.1 | — | — | — | — | — | — | — | 385.1 | — | — | — | 361.3 | — | 352.1 | — | — | — | — | 3.59 | — | — | — | — | 349.5 | 440.2 | 480 | |
| DynaSpecModel=Llama-3-8B-Instruct2026.04 | 378.1 | — | — | — | — | 382.9 | — | — | — | — | — | — | — | 353.3 | — | — | — | 350.2 | — | 333 | — | — | — | — | 3.78 | — | — | — | — | 344.2 | 429.7 | 453.2 | |
| FR-SpecModel=Llama-3-8B-Instruct2026.04 | 369.7 | — | — | — | — | 336.3 | — | — | — | — | — | — | — | 356 | — | — | — | 351 | — | 330.2 | — | — | — | — | 3.62 | — | — | — | — | 338.4 | 419.3 | 456.5 | |
| CORALModel=Llama-3-8B-Instruct2026.04 | 359.9 | — | — | — | — | 371.8 | — | — | — | — | — | — | — | 333.9 | — | — | — | 335.4 | — | 318.2 | — | — | — | — | 3.92 | — | — | — | — | 329.9 | 403 | 427.4 | |
| EAGLEModel=Llama-3-8B-Instruct2026.04 | 336.9 | — | — | — | — | 332 | — | — | — | — | — | — | — | 315 | — | — | — | 316.4 | — | 300.2 | — | — | — | — | 3.8 | — | — | — | — | 311.2 | 380.2 | 403.2 | |
| ARModel=Llama-3-8B-Instruct2026.04 | 172.6 | — | — | — | — | 162.3 | — | — | — | — | — | — | — | 174.3 | — | — | — | 174.2 | — | 165 | — | — | — | — | 1 | — | — | — | — | 175.7 | 178.5 | 178.2 | |
| CATS w/ EAGLEDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 4.18 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.1802 | — | — | — | — | — | — | — | |
| CATSDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.4791 | — | — | — | — | — | — | — | |
| CATS w/ EAGLEDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.5097 | — | — | — | — | — | — | — | |
| CATSDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.99 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.8816 | — | — | — | — | — | — | — | |
| ConFuTarget Model=Llama3.1-8B-Instruct, Nodes=60, Temperature (T)=0.02026.03 | 2.73 | 5.31 | 2.79 | 4.92 | 2.59 | 6.18 | 3.32 | 5.34 | 2.73 | 5.72 | 3.09 | 5.45 | 2.95 | 4.24 | 2.23 | 5.15 | 2.73 | 4.91 | 2.53 | 5.37 | 2.6 | 5.89 | 3.11 | 5.25 | — | — | — | — | — | — | — | — | |
| ConFuTarget Model=Llama3.1-8B-Instruct, Nodes=30, Temperature (T)=0.02026.03 | 2.69 | 5.03 | 2.74 | 4.64 | 2.5 | 6 | 3.32 | 5.19 | 2.75 | 5.47 | 3.03 | 5.18 | 2.89 | 4.07 | 2.2 | 4.87 | 2.66 | 4.63 | 2.46 | 5.19 | 2.6 | 5.65 | 3.08 | 5.01 | — | — | — | — | — | — | — | — | |
| Eagle3Target Model=Llama3.1-8B-Instruct, Nodes=60, Temperature (T)=0.02026.03 | 2.61 | 4.74 | 2.59 | 4.5 | 2.43 | 5.91 | 3.27 | 5.04 | 2.65 | 5.29 | 2.94 | 4.92 | 2.75 | 3.96 | 2.14 | 4.58 | 2.5 | 4.69 | 2.48 | 5.03 | 2.52 | 5.49 | 2.99 | 4.87 | — | — | — | — | — | — | — | — | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 2.53 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.3494 | — | — | — | — | — | — | — | |
| MedusaDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.4884 | — | — | — | — | — | — | — | |
| ConFuTarget Model=Llama3.1-8B-Instruct, Nodes=60, Temperature (T)=0.72026.03 | 2.48 | 5.16 | 2.62 | 4.65 | 2.37 | 5.9 | 3.04 | 5.13 | 2.53 | 5.35 | 2.75 | 5.11 | 2.65 | 4.05 | 2.03 | 4.85 | 2.46 | 4.49 | 2.24 | 5.07 | 2.38 | 5.61 | 2.83 | 4.96 | — | — | — | — | — | — | — | — | |
| ConFuTarget Model=Llama3.1-8B-Instruct, Nodes=30, Temperature (T)=0.72026.03 | 2.44 | 4.57 | 2.42 | 4.33 | 2.25 | 5.74 | 3.07 | 5.07 | 2.59 | 4.99 | 2.67 | 4.77 | 2.57 | 3.99 | 2.08 | 4.53 | 2.39 | 4.16 | 2.15 | 4.84 | 2.36 | 5.34 | 2.81 | 4.7 | — | — | — | — | — | — | — | — | |
| Eagle3Target Model=Llama3.1-8B-Instruct, Nodes=30, Temperature (T)=0.02026.03 | 2.36 | 4.39 | 2.43 | 4.26 | 2.31 | 5.65 | 3.15 | 4.89 | 2.59 | 5.02 | 2.8 | 4.6 | 2.59 | 3.76 | 2.04 | 4.28 | 2.35 | 4.35 | 2.32 | 4.78 | 1.59 | 5.16 | 2.82 | 4.59 | — | — | — | — | — | — | — | — | |
| Eagle3Target Model=Llama3.1-8B-Instruct, Nodes=60, Temperature (T)=0.72026.03 | 2.34 | 4.46 | 2.31 | 4.23 | 2.19 | 5.51 | 2.9 | 4.91 | 2.45 | 4.81 | 2.52 | 4.67 | 2.46 | 3.78 | 1.94 | 4.33 | 2.24 | 4.33 | 2.21 | 4.79 | 2.28 | 5.26 | 2.7 | 4.61 | — | — | — | — | — | — | — | — | |
| MedusaDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.3116 | — | — | — | — | — | — | — | |
| Eagle3Target Model=Llama3.1-8B-Instruct, Nodes=30, Temperature (T)=0.72026.03 | 2.31 | 4.09 | 2.23 | 3.91 | 2.14 | 5.36 | 2.95 | 4.72 | 2.48 | 4.45 | 2.45 | 4.27 | 2.34 | 3.64 | 1.96 | 4.03 | 2.19 | 3.99 | 2.12 | 4.49 | 2.27 | 4.88 | 2.63 | 4.31 | — | — | — | — | — | — | — | — | |
| LookaheadDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 2.29 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.2785 | — | — | — | — | — | — | — | |
| ConFuTarget Model=Llama3.1-8B-Instruct, Nodes=60, Temperature (T)=1.02026.03 | 2.18 | 4.48 | 2.26 | 4 | 2.02 | 5.34 | 2.73 | 4.85 | 2.36 | 4.18 | 2.14 | 4.01 | 2.05 | 3.79 | 1.89 | 4.34 | 2.19 | 3.77 | 1.88 | 4.4 | 2.09 | 5.04 | 2.54 | 4.37 | — | — | — | — | — | — | — | — | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 2.18 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.8943 | — | — | — | — | — | — | — | |
| ConFuTarget Model=Llama3.1-8B-Instruct, Nodes=30, Temperature (T)=1.02026.03 | 2.15 | 4.18 | 2.16 | 3.69 | 1.96 | 5.05 | 2.71 | 4.54 | 2.31 | 3.91 | 2.11 | 3.65 | 1.97 | 3.6 | 1.88 | 4.09 | 2.17 | 3.46 | 1.81 | 4.32 | 2.13 | 4.71 | 2.49 | 4.11 | — | — | — | — | — | — | — | — | |
| KangarooDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.7777 | — | — | — | — | — | — | — | |
| Eagle3Target Model=Llama3.1-8B-Instruct, Nodes=60, Temperature (T)=1.02026.03 | 2.09 | 3.87 | 2.03 | 3.49 | 1.82 | 4.68 | 2.48 | 4.41 | 2.2 | 4 | 2.12 | 3.6 | 1.92 | 3.57 | 1.83 | 3.97 | 2.07 | 3.75 | 1.91 | 4.13 | 2.01 | 4.73 | 2.46 | 4.07 | — | — | — | — | — | — | — | — | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.7487 | — | — | — | — | — | — | — | |
| Eagle3Target Model=Llama3.1-8B-Instruct, Nodes=30, Temperature (T)=1.02026.03 | 2.04 | 3.35 | 1.81 | 3.34 | 1.8 | 4.84 | 2.63 | 4.42 | 2.29 | 4.01 | 2.19 | 3.02 | 1.65 | 3.3 | 1.75 | 3.64 | 1.95 | 3.57 | 1.88 | 3.96 | 1.99 | 4.48 | 2.41 | 3.85 | — | — | — | — | — | — | — | — | |
| JoyAI-LLM FlashGPU=1, MTP layers=3, Concurrency=642026.04 | 1.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.2 | 40.35 | 66 | 4,241 | — | — | — | — | |
| GLM-5GPU=8, MTP layers=3, Concurrency=642026.04 | 1.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.03 | 75.84 | 31 | 1,969 | — | — | — | — | |
| DeepSeek-V3.2GPU=8, MTP layers=3, Concurrency=642026.04 | 1.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.55 | 63.72 | 31 | 1,958 | — | — | — | — | |
| KangarooDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 1.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.2736 | — | — | — | — | — | — | — | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 1.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.2995 | — | — | — | — | — | — | — | |
| RESTDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.66 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.6243 | — | — | — | — | — | — | — | |
| RESTDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.66 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.6236 | — | — | — | — | — | — | — | |
| Qwen3.5-35B-A3BGPU=1, MTP layers=3, Concurrency=642026.04 | 1.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.18 | 72.56 | 85 | 5,428 | — | — | — | — | |
| MiMo-V2-FlashGPU=4, MTP layers=3, Concurrency=642026.04 | 1.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.69 | 67.22 | 47 | 3,033 | — | — | — | — | |
| Step-3.5-FlashGPU=4, MTP layers=3, Concurrency=642026.04 | 1.39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.21 | 55.23 | 48 | 3,048 | — | — | — | — | |
| DeepSeek-V3GPU=8, MTP layers=3, Concurrency=642026.04 | 1.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.71 | 56.86 | 25 | 1,608 | — | — | — | — | |
| GLM-4.7-FlashGPU=1, MTP layers=3, Concurrency=642026.04 | 1.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.11 | 36.84 | 26 | 1,695 | — | — | — | — | |
| EXP3Model=Llama-3.1, Strategy=EXP32025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 52.19 | 4.04 | — | — | — | |
| EXP3Model=Qwen-3-8B, Strategy=EXP32025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.02 | 3.58 | — | — | — | |
| EXP3Model=Qwen-3-32B, Strategy=EXP32025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.86 | 3.43 | — | — | — | |
| HedgeSpecModel=Llama-3.1, Strategy=Hedge2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 60.8 | 5.31 | — | — | — | |
| HedgeSpecModel=Qwen-3-8B, Strategy=Hedge2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 50.7 | 4.7 | — | — | — | |
| HedgeSpecModel=Qwen-3-32B, Strategy=Hedge2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.73 | 3.97 | — | — | — | |
| UCBModel=Llama-3.1, Strategy=UCB2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 52.68 | 4.05 | — | — | — | |
| UCBModel=Qwen-3-8B, Strategy=UCB2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.23 | 3.65 | — | — | — | |
| UCBModel=Qwen-3-32B, Strategy=UCB2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.24 | 3.38 | — | — | — | |
| VanillaModel=Llama-3.1, Strategy=Vanilla2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 18.24 | 1 | — | — | — | |
| VanillaModel=Qwen-3-8B, Strategy=Vanilla2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 14.32 | 1 | — | — | — | |
| VanillaModel=Qwen-3-32B, Strategy=Vanilla2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 8.12 | 1 | — | — | — |