List your company
ResearchFormerly WizWand

Decoding Throughput on GSM8k

2,087Throughput (tok/s)FLARE-2B
-37.824513.8131,065.451,617.087Jun 1, 2026
Updated 5mo ago

Evaluation results

MethodLinks
FLARE-2BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.062,087—————
FLARE-2BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.061,314.7—————
FLARE-4BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.061,293.2—————
LLaDA-2.1-miniConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.061,224.1—————
FLARE-9BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.061,096.5—————
LLaDA-2.1-miniConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06963—————
FLARE-4BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06868.7—————
FLARE-9BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06696.1—————
LLaDA-2.1-miniConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06524.5—————
LLaDA-2.0-miniConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06518.7—————
FLARE-2BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06487—————
SDAR-1.7BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06438.3—————
SDAR-4BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06426.9—————
LLaDA-2.0-miniConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06408.7—————
SDAR-8BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06388.9—————
SDAR-1.7BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06376.6—————
SDAR-4BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06327.3—————
FLARE-4BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06292.1—————
SDAR-8BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06267.5—————
FLARE-9BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06217.4—————
SDAR-30B-A3BConcurrency level (C)=8, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06216.4—————
LLaDA-2.0-miniConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06185.6—————
SDAR-1.7BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06137—————
SDAR-30B-A3BConcurrency level (C)=4, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06124.6—————
SDAR-4BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.06102.8—————
SDAR-8BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.0683—————
SDAR-30B-A3BConcurrency level (C)=1, Hardware=1 x A100 80GB, Precision=bf16, Serving Stack=SGLang, max_new_tokens=2048, ignore_eos=true, temperature=1.0, top-p=0.95, top-k=502026.0643.9—————
AutoregressiveModel=Llama-3.1-Instruct 70B/1B2026.03—54.7————
AutoregressiveModel=Qwen-3 32B/0.6B2026.03—88.8————
Speculative DecodingModel=Llama-3.1-Instruct 70B/1B2026.03——188———
Speculative DecodingModel=Qwen-3 32B/0.6B2026.03——152———
SSDModel=Llama-3.1-Instruct 70B/1B2026.03———3011.65.5
SSDModel=Qwen-3 32B/0.6B2026.03———2341.542.64