ResearchTasksInference-side optimisationFollowBenchmarksDataset NameSOTA MethodSortMost resultsRecently updatedMost papersApplyDataset NameSOTA methodMetricTrendResultsLast UpdatedGeneral Inference-Side OptimizationVendor prompt caching0Per-Call Latency (ms)2May 12, 2026