# From independent patches to coordinated attention: Controlling information flow in vision transformers

Source: [From independent patches to coordinated attention: Controlling information flow in vision transformers](<https://www.sota2.com/research/paper/6984243ea8b01e76cc8ee8d1>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-06T05:00:14.101Z

Authors: Kieran A. Murphy

Published: 2026-02-04T17:33:24.000Z

## Abstract

We make the information transmitted by attention an explicit, measurable quantity in vision transformers. By inserting variational information bottlenecks on all attention-mediated writes to the residual stream -- without other architectural changes -- we train models with an explicit information cost and obtain a controllable spectrum from independent patch processing to fully expressive global attention. On ImageNet-100, we characterize how classification behavior and information routing evolve across this spectrum, and provide initial insights into how global visual representations emerge from local patch processing by analyzing the first attention heads that transmit information. By biasing learning toward solutions with constrained internal communication, our approach yields models that are more tractable for mechanistic analysis and more amenable to control.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2602.04784>)

[PDF](<https://arxiv.org/pdf/2602.04784.pdf>)