# Preference Redirection via Attention Concentration: An Attack on Computer Use Agents

Source: [Preference Redirection via Attention Concentration: An Attack on Computer Use Agents](<https://www.sota2.com/research/paper/69d884287d0172140ec77a9a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-08-12T05:00:21.403Z

Authors: Dominik Seip, Matthias Hein

Published: 2026-04-09T09:08:40.000Z

## Abstract

Advancements in multimodal foundation models have enabled the development of Computer Use Agents (CUAs) capable of autonomously interacting with GUI environments. As CUAs are not restricted to certain tools, they allow to automate more complex agentic tasks but at the same time open up new security vulnerabilities. While prior work has concentrated on the language modality, the vulnerability of the vision modality has received less attention. In this paper, we introduce PRAC, a novel attack that, unlike prior work targeting the VLM output directly, manipulates the model's internal preferences by redirecting its attention toward a stealthy adversarial patch. We show that PRAC is able to manipulate the selection process of a CUA on an online shopping platform towards a chosen target product. While we require white-box access to the model for the creation of the attack, we show that our attack generalizes to fine-tuned versions of the same model, presenting a critical threat as multiple companies build specific CUAs based on open weights models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Target Product Selection on Target Product Selection transfer to fine-tunes](<https://www.sota2.com/research/sota/target-product-selection-on-target-product-selection-transfer-to-fine-tunes.md>): Target Product Selection; Target Product Selection transfer to fine-tunes; Selection Success Rate: 95.5.

- [Target product selection on Webshop](<https://www.sota2.com/research/sota/target-product-selection-on-webshop.md>): Target Product Selection; Webshop; PRAC: 98.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.08005>)

[PDF](<https://arxiv.org/pdf/2604.08005.pdf>)