ResearchBenchmarksLLM Decoding on Llama-2-70BLLM Decoding on Llama-2-70BFollow0.2163Per-step Decoding LatencyPre30.2117120.2426810.273650.304619Jun 4, 2025Evaluation resultsMethodMethodLinksPer-step Decoding LatencyPre3Batchsize=1Batchsize=12025.060.2163Pre3Batchsize=4Batchsize=42025.060.2407XGrammarBatchsize=1Batchsize=12025.060.303XGrammarBatchsize=4Batchsize=42025.060.331