Decoding Throughput on Alpaca
88.8AR Throughput (tok/s)Autoregressive
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| AutoregressiveModel=Qwen-3 32B/0.6B2026.03 | 88.8 | — | — | — | — | |
| AutoregressiveModel=Llama-3.1-Instruct 70B/1B2026.03 | 54.7 | — | — | — | — | |
| Speculative DecodingModel=Llama-3.1-Instruct 70B/1B2026.03 | — | 145 | — | — | — | |
| Speculative DecodingModel=Qwen-3 32B/0.6B2026.03 | — | 127 | — | — | — | |
| SSDModel=Llama-3.1-Instruct 70B/1B2026.03 | — | — | 224 | 1.55 | 4.1 | |
| SSDModel=Qwen-3 32B/0.6B2026.03 | — | — | 185 | 1.47 | 2.08 |