ResearchTasksInteractive ServingInteractive ServingFollowBenchmarksDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyDatasetSOTA methodMetricTrendResultsLast updatedLlama 3.1-8B-Instruct 512 -> 128 tokens, concurrency=1Pre-comp. clustering rebuild269Throughput (Tok/s)6Mar 19, 2026
Llama 3.1-8B-Instruct 512 -> 128 tokens, concurrency=1Pre-comp. clustering rebuild269Throughput (Tok/s)6Mar 19, 2026