Decoding Throughput on GSM8k
2,087Throughput (tok/s)FLARE-2B
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FLARE-2BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 2,087 | — | — | — | — | — | |
| FLARE-2BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 1,314.7 | — | — | — | — | — | |
| FLARE-4BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 1,293.2 | — | — | — | — | — | |
| LLaDA-2.1-miniConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 1,224.1 | — | — | — | — | — | |
| FLARE-9BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 1,096.5 | — | — | — | — | — | |
| LLaDA-2.1-miniConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 963 | — | — | — | — | — | |
| FLARE-4BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 868.7 | — | — | — | — | — | |
| FLARE-9BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 696.1 | — | — | — | — | — | |
| LLaDA-2.1-miniConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 524.5 | — | — | — | — | — | |
| LLaDA-2.0-miniConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 518.7 | — | — | — | — | — | |
| FLARE-2BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 487 | — | — | — | — | — | |
| SDAR-1.7BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 438.3 | — | — | — | — | — | |
| SDAR-4BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 426.9 | — | — | — | — | — | |
| LLaDA-2.0-miniConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 408.7 | — | — | — | — | — | |
| SDAR-8BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 388.9 | — | — | — | — | — | |
| SDAR-1.7BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 376.6 | — | — | — | — | — | |
| SDAR-4BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 327.3 | — | — | — | — | — | |
| FLARE-4BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 292.1 | — | — | — | — | — | |
| SDAR-8BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 267.5 | — | — | — | — | — | |
| FLARE-9BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 217.4 | — | — | — | — | — | |
| SDAR-30B-A3BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 216.4 | — | — | — | — | — | |
| LLaDA-2.0-miniConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 185.6 | — | — | — | — | — | |
| SDAR-1.7BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 137 | — | — | — | — | — | |
| SDAR-30B-A3BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 124.6 | — | — | — | — | — | |
| SDAR-4BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 102.8 | — | — | — | — | — | |
| SDAR-8BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 83 | — | — | — | — | — | |
| SDAR-30B-A3BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06 | 43.9 | — | — | — | — | — | |
| AutoregressiveModel=Llama-3.1-Instruct 70B/1B2026.03 | — | 54.7 | — | — | — | — | |
| AutoregressiveModel=Qwen-3 32B/0.6B2026.03 | — | 88.8 | — | — | — | — | |
| Speculative DecodingModel=Llama-3.1-Instruct 70B/1B2026.03 | — | — | 188 | — | — | — | |
| Speculative DecodingModel=Qwen-3 32B/0.6B2026.03 | — | — | 152 | — | — | — | |
| SSDModel=Llama-3.1-Instruct 70B/1B2026.03 | — | — | — | 301 | 1.6 | 5.5 | |
| SSDModel=Qwen-3 32B/0.6B2026.03 | — | — | — | 234 | 1.54 | 2.64 |