Mathematical Reasoning on Math500 (Throughput, Speedup)
3,636.1ThroughputSpecBundle
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SpecBundleTarget Model=Qwen-30B-A3B, Draft Model=SpecBundle, #GPUs=42026.03 | 3,636.1 | 1.48 | |
| SpecBundleTarget Model=Llama-4-Scout, Draft Model=SpecBundle, #GPUs=82026.03 | 2,110.3 | 3.76 | |
| SpecBundleTarget Model=Ling-Flash-V2, Draft Model=SpecBundle, #GPUs=82026.03 | 1,685.6 | 2.21 | |
| SpecBundleTarget Model=Llama-3.3-70B, Draft Model=SpecBundle, #GPUs=42026.03 | 1,524.9 | 2.69 | |
| Existing Speculative DecodingTarget Model=Llama-4-Scout, Draft Model=Existing, #GPUs=82026.03 | 1,479 | 2.63 | |
| AutoregressiveTarget Model=Qwen-30B-A3B, Draft Model=None, #GPUs=42026.03 | 1,469 | 1 | |
| SpecBundleTarget Model=Qwen-235B-A22B, Draft Model=SpecBundle, #GPUs=82026.03 | 1,399.2 | 2.38 | |
| Existing Speculative DecodingTarget Model=Llama-3.3-70B, Draft Model=Existing, #GPUs=42026.03 | 1,122.2 | 1.98 | |
| SpecBundleTarget Model=Kimi-K2, Draft Model=SpecBundle, #GPUs=82026.03 | 1,022.7 | 2.08 | |
| Existing Speculative DecodingTarget Model=Qwen-235B-A22B, Draft Model=Existing, #GPUs=82026.03 | 821.8 | 1.39 | |
| AutoregressiveTarget Model=Ling-Flash-V2, Draft Model=None, #GPUs=82026.03 | 762.7 | 1 | |
| SpecBundleTarget Model=Llama-3.1-8B, Draft Model=SpecBundle, #GPUs=12026.03 | 638 | 3.34 | |
| AutoregressiveTarget Model=Qwen-235B-A22B, Draft Model=None, #GPUs=82026.03 | 587.4 | 1 | |
| AutoregressiveTarget Model=Llama-3.3-70B, Draft Model=None, #GPUs=42026.03 | 567.4 | 1 | |
| AutoregressiveTarget Model=Llama-4-Scout, Draft Model=None, #GPUs=82026.03 | 561.8 | 1 | |
| AutoregressiveTarget Model=Kimi-K2, Draft Model=None, #GPUs=82026.03 | 492.1 | 1 | |
| Existing Speculative DecodingTarget Model=Llama-3.1-8B, Draft Model=Existing, #GPUs=12026.03 | 422.4 | 2.21 | |
| AutoregressiveTarget Model=Llama-3.1-8B, Draft Model=None, #GPUs=12026.03 | 191 | 1 |