# SALLIE: Generation-Free Hidden-State Detection of Jailbreaks and Prompt Injections Across Text and Vision

Source: [SALLIE: Generation-Free Hidden-State Detection of Jailbreaks and Prompt Injections Across Text and Vision](<https://www.sota2.com/research/paper/69d732667d0172140ec7698b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-08-12T05:00:21.403Z

Authors: Guy Azov, Ofer Rivlin, Guy Shtar

Published: 2026-04-06T16:29:05.000Z

## Abstract

Large Language Models (LLMs) and Vision-Language Models (VLMs) are vulnerable to jailbreaks and prompt injections delivered through text or images. Existing defenses often narrow threat coverage or add inference cost through input transformations, repeated model queries, generated responses, or threat-specific detectors. We present SALLIE (Single-Pass Activation Lookup for Layerwise Input Evaluation), a white-box, generation-free detector with a common architecture for textual and visual jailbreaks and prompt injections. Using one backbone forward pass, SALLIE extracts last-token residual-stream representations, applies layer-wise k-nearest-neighbor probes, and averages their scores across a contiguous layer range, without modifying the backbone or generating a response. To distinguish reusable detector configurations from deployment-specific tuning, we compare shared, threshold-only, and fully calibrated regimes across backbones and modalities. We evaluate SALLIE on compact open-weight VLMs (Gemma-3-4b-it, Phi-3.5-vision-instruct, SmolVLM2-2.2B-Instruct) using heterogeneous held-out test data spanning both threat families and modalities. Fully calibrated SALLIE-Phi achieves an aggregate F1 of 0.87 and higher aggregate balanced accuracy than RCS-KCD on all three backbones under a matched-calibration protocol. On Phi-3.5 visual inputs, threshold-only calibration yields an F1 of 0.99 with zero observed false positives, versus 0.54 for zero-shot Gemini-2.5-Flash-Lite. On Gemma-4-E2B-it, a backbone held out from representation-hyperparameter selection, the same procedure raises F1 from 0.62 to 0.79 on text and 0.68 to 0.98 on visual inputs. Representation hyperparameters can thus transfer across some backbones, while operating points remain backbone-, modality-, and distribution-dependent. We further report FPR/FNR tradeoffs, latency, and representation ablations.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multimodal Attack Detection on Multimodal attack detection (test)](<https://www.sota2.com/research/sota/multimodal-attack-detection-on-multimodal-attack-detection-test.md>): Multimodal Attack Detection; Multimodal attack detection (test); Precision: 99.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.06247>)

[PDF](<https://arxiv.org/pdf/2604.06247.pdf>)
