ResearchTasksLarge Language Model ServingLarge Language Model ServingFollowBenchmarksDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyDatasetSOTA methodMetricTrendResultsLast updatedvLLM serving benchmark 128 prompts, 32 pre-fill tokens, 256 generation tokensDeInfer76TTFT (ms)9Apr 21, 2026
vLLM serving benchmark 128 prompts, 32 pre-fill tokens, 256 generation tokensDeInfer76TTFT (ms)9Apr 21, 2026