ResearchDatasetsContext sequencesContext sequencesFollowBenchmarksTaskDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyTaskDatasetSOTA resultTrendResultsTime to First Token (TTFT) measurementContext sequences 32K-256K vLLM 0.18.0 / 0.20.0 (online-serving)2.944TTFT (Dense, s)7
Time to First Token (TTFT) measurementContext sequences 32K-256K vLLM 0.18.0 / 0.20.0 (online-serving)2.944TTFT (Dense, s)7