# LipCache: A Local Inference Proxy with Certified Caching for Edge Image Classification Service

Source: [LipCache: A Local Inference Proxy with Certified Caching for Edge Image Classification Service](<https://www.sota2.com/research/paper/6a7ea14a4b236a9bba1a2d13>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-08-15T05:00:08.502Z

Authors: Zhengzhe Xiang, Yinlin Chen, Fuli Ying, Binbin Zhou, Hailiang Zhao, Schahram Dustdar

Published: 2026-08-13T12:16:58.000Z

## Abstract

As edge-side vision services continue to expand toward low-latency, high-throughput scenarios, reducing the inference cost of vision models without sacrificing reliability has become a central concern. Existing semantic caching methods largely rely on empirical similarity thresholds; while such thresholds improve hit rates, they tend to introduce silent misclassifications near decision boundaries. To address this, we propose \\texttt{LipCache}, a certified semantic caching framework for image classification. Without modifying the existing deployed main model, \\texttt{MainNet}, the framework introduces a lightweight network, \\texttt{GuardNet}, that maps inputs into a low-dimensional feature space subject to a Lipschitz constraint. It then computes a per-sample certified reuse radius from the local classification margin and the spectral norm of the classification head. At runtime, a cached result is reused only when the query feature falls inside the certified reuse ball; otherwise, the query falls back to \\texttt{MainNet}. Thus, cache hits are transformed from empirical threshold tests into geometric certification decisions with explicit theoretical boundaries. Across standard image classification tasks like CIFAR, Tiny-ImageNet, and SVHN, \\texttt{LipCache} achieves a measured speedup of up to $1.65\\times$ with limited end-to-end accuracy degradation, while all accepted cache hits satisfy the \\texttt{GuardNet}-side certified-consistency condition. Furthermore, an enhanced \\texttt{GuardNet} training recipe substantially improves cache hit rates in the Tiny-ImageNet multi-class extension while maintaining a certified-consistency rate of $100\\%$. These results demonstrate that per-sample certified reuse can reduce main-model fallback while preserving theoretical consistency, providing a feasible approach to reliable cache-assisted inference at the edge.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2608.13144>)

[PDF](<https://arxiv.org/pdf/2608.13144.pdf>)