Semantic Caching
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
70Static Origin Served Fraction
13
Feb 26, 2026
1,504.43End-to-End Latency (min)
4
May 26, 2026
1,866.58End-to-End Latency (min)
4
May 26, 2026
383.32Latency (min)
4
May 26, 2026
6,345.61Cumulative End-to-End Inference Latency (minutes)
4
May 26, 2026