ResearchDatasetsControlled tracesFollowBenchmarksTask NameDataset NameSOTA MethodSortMost resultsRecently updatedMost papersApplyTask NameDataset NameSOTA ResultTrendResultsGPU serving throughputControlled traces 8 streams 4096 prompt tokens 512 decode tokens274.61Throughput (tok/s)6
GPU serving throughputControlled traces 8 streams 4096 prompt tokens 512 decode tokens274.61Throughput (tok/s)6