# The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP

Source: [The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP](<https://www.sota2.com/research/paper/6a167aabdbd7150506f800ab>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-28T05:00:05.405Z

Authors: Kahyeon Nam, Hyesong Choi

Published: 2026-05-26T00:45:46.000Z

## Abstract

Deploying Vision-Language Models on resource-constrained hardware typically requires INT8 quantization, but in joint-embedding architectures such as CLIP this introduces a failure mode distinct from quantized CNN classifiers: activation noise accumulated across transformer blocks perturbs the direction of the multimodal embedding, eroding the cosine alignment on which zero-shot retrieval depends. We characterize this as Quantization-Induced Representation Collapse (QIRC) and quantify it on INT8 CLIP ViT-B/32, where the layer-wise noise-to-signal ratio grows from below 10% in shallow blocks to 52% at Layer 11. We propose LRA-EE (Layer-wise Representation-Aware Early Exit), which bypasses noise-saturated deep layers via Spatio-Semantic Aggregation (replacing the immature shallow \[CLS\] with a global patch-token average), a learned multi-feature gate (confidence, top-2 margin, spatial-activation variance), and Layer-adaptive Confidence Thresholding calibrated to each layer's Information-to-Noise Ratio. On ImageNet-1K zero-shot classification, LRA-EE reduces FLOPs by 13.4% and improves Top-1 accuracy by +2.44%p (58.72% -\> 61.16%) over the INT8 baseline. A four-quadrant decomposition isolates the Rescue Effect: 9.5% of samples are correctly classified at shallow exits but lost to noise at full depth, against only 7.1% suffering the inverse.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet-1K (val) (Accuracy and FLOPs Saving)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-val-accuracy-and-flops-saving.md>): Image Classification; ImageNet-1k (val); FLOPs Saving (%): 13.38.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.26415>)

[PDF](<https://arxiv.org/pdf/2605.26415.pdf>)