List your company
ResearchFormerly WizWand

End-to-end single-step decoding on DeepSeek-R1-Distill-LLaMA-8B 64K Context

24.1Latency (ms)LessIsMore
23.68826.46929.2532.031Aug 9, 2025
Updated 6mo ago

Evaluation results

MethodLinks
LessIsMoreToken Budget=2K, Serving Stack=SGLang + FlashInfer, Hardware=NVIDIA A5000 GPU2025.0824.1
QuestToken Budget=2K, Serving Stack=SGLang + FlashInfer, Hardware=NVIDIA A5000 GPU2025.0824.8
TidalDecodeToken Budget=2K, Serving Stack=SGLang + FlashInfer, Hardware=NVIDIA A5000 GPU2025.0825.4
Full AttentionToken Budget=2K, Serving Stack=SGLang + FlashInfer, Hardware=NVIDIA A5000 GPU2025.0834.4