# BRo-JEPA: Learning Modular Arithmetic in Latent Space

Source: [BRo-JEPA: Learning Modular Arithmetic in Latent Space](<https://www.sota2.com/research/paper/6a1e6410dbd7150506f856a7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-02T05:02:08.515Z

Authors: Divyansh Jha, Yuanfang Xie, Varan Mehra, Brennen Yu

Published: 2026-05-31T18:01:43.000Z

## Abstract

Can neural networks learn abstract algebraic rules, or do they merely memorize training patterns? We investigate this using MNIST digits as states and modular arithmetic operations as actions in a JEPA-style latent world model. Standard supervised baselines and JEPA models with additive operation embeddings fit seen operations but fail to extrapolate reliably to unseen ones. To bridge this gap, we introduce a block-rotation predictor that imposes the circular structure of modulo-10 arithmetic in latent space. This enables strong zero-shot generalization, with the best ResNet-based JEPA block-rotation model achieving 99.46\\% zero-shot and 99.46\\% rollout accuracy. Our results suggest that latent world models can learn symbolic transformation rules when architecture matches the structure of the problem. Our code can be \\href{https://github.com/DL-World-Models/mnist-math}{accessed here}.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Modular arithmetic prediction on MNIST modular arithmetic (train)](<https://www.sota2.com/research/sota/modular-arithmetic-prediction-on-mnist-modular-arithmetic-train.md>): Modular arithmetic prediction; MNIST modular arithmetic (train); Train Accuracy: 99.94.

- [Modular arithmetic prediction on MNIST modular arithmetic Unseen operations (test)](<https://www.sota2.com/research/sota/modular-arithmetic-prediction-on-mnist-modular-arithmetic-unseen-operations-test.md>): Modular arithmetic prediction; MNIST modular arithmetic Unseen operations (test); Zero-Shot Op Accuracy: 99.46.

- [Modular arithmetic prediction on MNIST modular arithmetic Seen operations (test)](<https://www.sota2.com/research/sota/modular-arithmetic-prediction-on-mnist-modular-arithmetic-seen-operations-test.md>): Modular arithmetic prediction; MNIST modular arithmetic Seen operations (test); Seen Op Accuracy: 99.45.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.01372>)

[PDF](<https://arxiv.org/pdf/2606.01372.pdf>)
