Text Generation on Alpaca (Efficiency)
5.68Speedup RatioHybrid Verified Decoding
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Hybrid Verified DecodingTarget Model=Llama3.1-8B, Reference Baseline=SuffixDecoding2026.05 | 5.68 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Qwen3-8B, Reference Baseline=SuffixDecoding2026.05 | 3.84 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Llama3.1-8B, Reference Baseline=Greedy Decoding2026.05 | 3.14 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Qwen3-8B, Reference Baseline=Greedy Decoding2026.05 | 2.73 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Qwen3-4B, Reference Baseline=SuffixDecoding2026.05 | 2.4 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Qwen3-4B, Reference Baseline=Greedy Decoding2026.05 | 2.06 | — | — | — | — | — | — | — | |
| GSDTarget Model=BLOOM-7b1, Draft Model=BLOOM-560m, k (maximum out-degree)=4, tau (threshold for redundant node)=1, theta_prob=0.4, theta_sib=0.12024.07 | 1.48 | 81.9 | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Qwen3-8B, Reference Baseline=Frequency-Based Hybrid Decoding2026.05 | 1.4 | — | — | — | — | — | — | — | |
| SSDTarget Model=BLOOM-7b1, Draft Model=BLOOM-560m, k (maximum out-degree)=4, tau (threshold for redundant node)=1, theta_prob=0.4, theta_sib=0.12024.07 | 1.12 | 62.8 | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Llama3.1-8B, Reference Baseline=Frequency-Based Hybrid Decoding2026.05 | 1.05 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Qwen3-8B, Reference Baseline=EAGLE32026.05 | 0.97 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Llama3.1-8B, Reference Baseline=EAGLE32026.05 | 0.96 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Qwen3-4B, Reference Baseline=EAGLE32026.05 | 0.85 | — | — | — | — | — | — | — | |
| Hybrid Verified DecodingTarget Model=Qwen3-4B, Reference Baseline=Frequency-Based Hybrid Decoding2026.05 | 0.83 | — | — | — | — | — | — | — | |
| TSDTarget Model=BLOOM-7b1, Draft Model=BLOOM-560m, k (maximum out-degree)=4, tau (threshold for redundant node)=1, theta_prob=0.4, theta_sib=0.12024.07 | 0.44 | 78.3 | — | — | — | — | — | — | |
| ARBackbone=Qwen3-4B, Temperature=1.02026.06 | — | — | — | 30.66 | — | — | — | — | |
| ARBackbone=Qwen3-8B, Temperature=1.02026.06 | — | — | — | 31.67 | — | — | — | — | |
| CADDTREEBackbone=Qwen3-4B, Temperature=1.02026.06 | — | — | — | 11.2 | 4.63 | — | 261 | — | |
| CADDTREEBackbone=Qwen3-8B, Temperature=1.02026.06 | — | — | — | 11.55 | 4.54 | — | 186 | — | |
| DDDTemperature=0, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 109.68 | — | — | — | — | — | |
| DDDTemperature=1, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 108.91 | — | — | — | — | — | |
| DDTree-oracleBackbone=Qwen3-4B, Temperature=1.02026.06 | — | — | — | 10.96 | 4.63 | 256 | — | — | |
| DDTree-oracleBackbone=Qwen3-8B, Temperature=1.02026.06 | — | — | — | 11.64 | 4.44 | 128 | — | — | |
| DFlashBackbone=Qwen3-4B, Temperature=1.02026.06 | — | — | — | 16.71 | 2.98 | — | — | — | |
| DFlashBackbone=Qwen3-8B, Temperature=1.02026.06 | — | — | — | 17.47 | 2.94 | — | — | — | |
| EAGLE-3Temperature=0, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 120.22 | — | — | — | — | — | |
| EAGLE-3Temperature=1, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 119.65 | — | — | — | — | — | |
| EVICTTemperature=0, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 143.73 | — | — | — | — | — | |
| EVICTTemperature=1, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 140.46 | — | — | — | — | — | |
| LookaheadTemperature=0, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 59.53 | — | — | — | — | — | |
| LookaheadTemperature=1, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 59.52 | — | — | — | — | — | |
| VanillaTemperature=0, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 82.69 | — | — | — | — | — | |
| VanillaTemperature=1, Backbone=Qwen3-235B-A22B, Inference Engine=SGLang, Hardware=NVIDIA A100 SXM4 80GB GPUs, Batch Size=1, Tensor Parallelism=TP=82026.05 | — | — | 81.99 | — | — | — | — | — |