Mathematical Reasoning on GSM8K (Throughput, Speedup)
1,499.6ThroughputSpecBundle
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecBundleTarget Model=Qwen-30B-A3B, Draft Model=SpecBundle, #GPUs=42026.03 | 1,499.6 | 1.4 | |
| SpecBundleTarget Model=Ling-Flash-V2, Draft Model=SpecBundle, #GPUs=82026.03 | 1,323 | 1.96 | |
| AutoregressiveTarget Model=Qwen-30B-A3B, Draft Model=None, #GPUs=42026.03 | 1,071.3 | 1 | |
| SpecBundleTarget Model=Llama-4-Scout, Draft Model=SpecBundle, #GPUs=82026.03 | 971.9 | 2.13 | |
| Existing Speculative DecodingTarget Model=Llama-4-Scout, Draft Model=Existing, #GPUs=82026.03 | 816.6 | 1.79 | |
| SpecBundleTarget Model=Qwen-235B-A22B, Draft Model=SpecBundle, #GPUs=82026.03 | 758.3 | 1.62 | |
| SpecBundleTarget Model=Llama-3.3-70B, Draft Model=SpecBundle, #GPUs=42026.03 | 722 | 1.59 | |
| Existing Speculative DecodingTarget Model=Qwen-235B-A22B, Draft Model=Existing, #GPUs=82026.03 | 697 | 1.49 | |
| AutoregressiveTarget Model=Ling-Flash-V2, Draft Model=None, #GPUs=82026.03 | 674.3 | 1 | |
| SpecBundleTarget Model=Kimi-K2, Draft Model=SpecBundle, #GPUs=82026.03 | 544.2 | 1.61 | |
| Existing Speculative DecodingTarget Model=Llama-3.3-70B, Draft Model=Existing, #GPUs=42026.03 | 521.5 | 1.15 | |
| AutoregressiveTarget Model=Qwen-235B-A22B, Draft Model=None, #GPUs=82026.03 | 469.1 | 1 | |
| AutoregressiveTarget Model=Llama-4-Scout, Draft Model=None, #GPUs=82026.03 | 455.9 | 1 | |
| AutoregressiveTarget Model=Llama-3.3-70B, Draft Model=None, #GPUs=42026.03 | 453.2 | 1 | |
| AutoregressiveTarget Model=Kimi-K2, Draft Model=None, #GPUs=82026.03 | 337.9 | 1 | |
| SpecBundleTarget Model=Llama-3.1-8B, Draft Model=SpecBundle, #GPUs=12026.03 | 329.7 | 1.81 | |
| Existing Speculative DecodingTarget Model=Llama-3.1-8B, Draft Model=Existing, #GPUs=12026.03 | 228.6 | 1.26 | |
| AutoregressiveTarget Model=Llama-3.1-8B, Draft Model=None, #GPUs=12026.03 | 181.8 | 1 |