# Follow My Eyes: Backdoor Attacks on Goal-Directed Scanpath Prediction Source: [Follow My Eyes: Backdoor Attacks on Goal-Directed Scanpath Prediction]() Collection index: [llms.txt]() Data updated: 2026-07-04T05:01:04.041Z Authors: Diana Romero, Mutahar Ali, Momin Ahmad Khan, Habiba Farrukh, Fatima Anwar, Salma Elmalaki Published: 2026-04-09T21:06:19.000Z ## Abstract Scanpath prediction models forecast the sequence of fixations a person makes while searching a scene, and increasingly serve as the upstream perception layer for foveated rendering, intent inference, and gaze-driven assistive interfaces. Since eye-tracking data is expensive to collect, these models are routinely fine-tuned from public datasets or third-party pretrained weights, exposing them to training-time poisoning. We present the first backdoor attacks on multimodal scanpath prediction. This task differs from classification: its output is a continuous, variable-length sequence of fixations, opening new avenues of attack. A fixed-trajectory backdoor is easy to implant, but it clusters poisoned samples away from clean data, making it detectable. We instead design two backdoor attacks that condition the malicious supervision on each scene to keep triggered outputs diverse and plausible: a spatial misdirection attack that redirects the predicted search toward an attacker-chosen object instead of the queried one, and a duration inflation attack that lengthens the predicted search by inserting extra fixations while preserving correct localization. Our attacks succeed across visual, textual, and multimodal triggers, with duration inflation reaching up to 93.5% attack success from as few as 540 poisoned samples (2.5% of training data), and spatial misdirection redirecting the search in up to 61% of triggered inputs. We evaluate our attacks against five existing backdoor defenses, spanning fine-tuning, fine-pruning, neural attention distillation, contrastive learning, and trigger inversion, and show none removes the backdoor without degrading model output below the usable threshold. Our attacks generalize across models and datasets, showing scanpath prediction models are vulnerable to backdoor attacks through data poisoning, and that designing an effective defense remains an open problem. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Scanpath Prediction Dataset](): Scanpath Prediction; Scanpath Prediction Dataset; Clean SS: 49.6. - [Scanpath Prediction on Visual Search Dataset Clean Inputs](): Scanpath Prediction; Visual Search Dataset Clean Inputs; BBox Hit Ratio: 83.5. - [Scanpath Prediction on Visual Search Dataset Poisoned](): Scanpath Prediction; Visual Search Dataset Poisoned; BBox Hit Ratio: 53.8. - [Scanpath Prediction on COCO-Search18 (test)](): Scanpath Prediction; COCO-Search18 (test); Scanpath Accuracy (R@1): not recorded. ## Original sources and implementation [Original paper]() [PDF]()