# Weight Feedback Computes the Jacobian Transpose Locally in Modern Deep Networks

Source: [Weight Feedback Computes the Jacobian Transpose Locally in Modern Deep Networks](<https://www.sota2.com/research/paper/6a58656d32fb9373148789ec>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-17T05:00:05.623Z

Authors: Junlong Shen, Xingyu Li

Published: 2026-07-15T02:18:23.000Z

## Abstract

Predictive Coding (PC) offers a biologically motivated alternative to backpropagation via local weight updates, yet routing error between layers still relies on an autograd Jacobian-transpose ($J^\\top$) product - the last non-local operation in PC. We show that this dependency is largely avoidable. For any layer $f(x)=\\mathrm{Act}(\\mathrm{Norm}(L(x)))$ with frozen normalization statistics, the exact $J^\\top$ factors into three locally available terms, $J^\\top v = L^\\top(s \\odot \\sigma'(z) \\odot v)$, where $\\sigma'$ is the activation derivative, $z$ is the pre-activation, and $s=\\gamma/\\sigma\_{\\mathrm{run}}$ is the normalization gain. Prior weight-feedback methods omitted both corrections; restoring them closes the transport gap for this layer class. Locality here holds up to three assumptions, which we state upfront: weight symmetry ($L^\\top$ mirrors the forward operator, as assumed by all PC), a soft spectral-norm control that is not synapse-local, and a nearest-neighbour approximation for MaxPool. Substituting the identity into PC yields WF-Act-PC, which removes the autograd backward pass from error transport. On CIFAR-10/100 (50 epochs, 5 seeds), WF-Act-PC is the only PC method whose accuracy improves with depth, surpassing iPC - the strongest classical PC baseline - by 2.7-22.3 pp on CIFAR-10. With both methods tuned per architecture, it matches or exceeds a comparably-tuned backpropagation baseline on the deeper CIFAR-10 architectures (VGG-9: 93.57% vs. 92.43%; ResNet-18: 92.76% vs. 91.54%) and on the harder Tiny-ImageNet benchmark, while trailing tuned BP on the deeper CIFAR-100 VGG cells. Our WF-Act-PC implementation is publicly available at https://github.com/jlshen025/pcax

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2607.13380>)

[PDF](<https://arxiv.org/pdf/2607.13380.pdf>)
