Coding on LiveCodeBench (Throughput/Speedup)
3,413ThroughputSpecBundle
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SpecBundleTarget Model=Qwen-30B-A3B, Draft Model=SpecBundle, #GPUs=42026.03 | 3,413 | 2.29 | |
| SpecBundleTarget Model=Llama-4-Scout, Draft Model=SpecBundle, #GPUs=82026.03 | 2,170.2 | 4.48 | |
| Existing Speculative DecodingTarget Model=Llama-4-Scout, Draft Model=Existing, #GPUs=82026.03 | 1,601.3 | 3.31 | |
| AutoregressiveTarget Model=Qwen-30B-A3B, Draft Model=None, #GPUs=42026.03 | 1,492.6 | 1 | |
| SpecBundleTarget Model=Llama-3.3-70B, Draft Model=SpecBundle, #GPUs=42026.03 | 1,459.4 | 2.6 | |
| SpecBundleTarget Model=Ling-Flash-V2, Draft Model=SpecBundle, #GPUs=82026.03 | 1,366.4 | 1.77 | |
| Existing Speculative DecodingTarget Model=Llama-3.3-70B, Draft Model=Existing, #GPUs=42026.03 | 1,303.4 | 2.32 | |
| SpecBundleTarget Model=Qwen-235B-A22B, Draft Model=SpecBundle, #GPUs=82026.03 | 1,155.7 | 1.93 | |
| SpecBundleTarget Model=Kimi-K2, Draft Model=SpecBundle, #GPUs=82026.03 | 904.4 | 1.81 | |
| Existing Speculative DecodingTarget Model=Qwen-235B-A22B, Draft Model=Existing, #GPUs=82026.03 | 803.8 | 1.34 | |
| AutoregressiveTarget Model=Ling-Flash-V2, Draft Model=None, #GPUs=82026.03 | 770.4 | 1 | |
| AutoregressiveTarget Model=Qwen-235B-A22B, Draft Model=None, #GPUs=82026.03 | 598.2 | 1 | |
| AutoregressiveTarget Model=Llama-3.3-70B, Draft Model=None, #GPUs=42026.03 | 560.9 | 1 | |
| SpecBundleTarget Model=Llama-3.1-8B, Draft Model=SpecBundle, #GPUs=12026.03 | 516.9 | 2.72 | |
| AutoregressiveTarget Model=Kimi-K2, Draft Model=None, #GPUs=82026.03 | 500.1 | 1 | |
| AutoregressiveTarget Model=Llama-4-Scout, Draft Model=None, #GPUs=82026.03 | 484.3 | 1 | |
| Existing Speculative DecodingTarget Model=Llama-3.1-8B, Draft Model=Existing, #GPUs=12026.03 | 398.4 | 2.1 | |
| AutoregressiveTarget Model=Llama-3.1-8B, Draft Model=None, #GPUs=12026.03 | 189.7 | 1 |