Coding on HumanEval (Throughput and Speedup)
3,070ThroughputSpecBundle
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecBundleTarget Model=Qwen-30B-A3B, Draft Model=SpecBundle, #GPUs=42026.03 | 3,070 | 2.25 | |
| SpecBundleTarget Model=Llama-4-Scout, Draft Model=SpecBundle, #GPUs=82026.03 | 1,944.8 | 3.08 | |
| Existing Speculative DecodingTarget Model=Llama-4-Scout, Draft Model=Existing, #GPUs=82026.03 | 1,556.5 | 2.46 | |
| SpecBundleTarget Model=Llama-3.3-70B, Draft Model=SpecBundle, #GPUs=42026.03 | 1,506 | 2.68 | |
| AutoregressiveTarget Model=Qwen-30B-A3B, Draft Model=None, #GPUs=42026.03 | 1,366.6 | 1 | |
| SpecBundleTarget Model=Ling-Flash-V2, Draft Model=SpecBundle, #GPUs=82026.03 | 1,359 | 1.83 | |
| Existing Speculative DecodingTarget Model=Llama-3.3-70B, Draft Model=Existing, #GPUs=42026.03 | 1,282.8 | 2.29 | |
| SpecBundleTarget Model=Qwen-235B-A22B, Draft Model=SpecBundle, #GPUs=82026.03 | 1,267.5 | 2.29 | |
| SpecBundleTarget Model=Kimi-K2, Draft Model=SpecBundle, #GPUs=82026.03 | 897.9 | 1.93 | |
| Existing Speculative DecodingTarget Model=Qwen-235B-A22B, Draft Model=Existing, #GPUs=82026.03 | 889.9 | 1.61 | |
| AutoregressiveTarget Model=Ling-Flash-V2, Draft Model=None, #GPUs=82026.03 | 740.2 | 1 | |
| AutoregressiveTarget Model=Llama-4-Scout, Draft Model=None, #GPUs=82026.03 | 631.9 | 1 | |
| SpecBundleTarget Model=Llama-3.1-8B, Draft Model=SpecBundle, #GPUs=12026.03 | 571.5 | 2.99 | |
| AutoregressiveTarget Model=Llama-3.3-70B, Draft Model=None, #GPUs=42026.03 | 561 | 1 | |
| AutoregressiveTarget Model=Qwen-235B-A22B, Draft Model=None, #GPUs=82026.03 | 553.1 | 1 | |
| Existing Speculative DecodingTarget Model=Llama-3.1-8B, Draft Model=Existing, #GPUs=12026.03 | 480.3 | 2.52 | |
| AutoregressiveTarget Model=Kimi-K2, Draft Model=None, #GPUs=82026.03 | 466.1 | 1 | |
| AutoregressiveTarget Model=Llama-3.1-8B, Draft Model=None, #GPUs=12026.03 | 190.9 | 1 |