# The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models

Source: [The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models](<https://www.sota2.com/research/paper/6a2106a8c10484bac23f033c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-08T06:00:01.208Z

Authors: Liuyuan Wen, Xun Zhu, Lihao Huang, Wenbin Li, Yang Gao

Published: 2026-05-29T10:55:50.000Z

## Abstract

Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output. By analyzing the residual stream geometry during multi-operand addition, we identify the Iso-Raw-Sum Trajectory (IRST), a geometric structure where representations are anchored by semantic digits and modulated by continuous carry fibers. We propose the Noisy Quantization Model to explain this geometry, framing arithmetic errors as Geometric Slippages caused by internal neural noise pushing a continuous, latent Carry Potential across quantization thresholds. This geometric framework further elucidates Probe Versatility, explaining how lightweight probes can disentangle coexisting latent signals (such as ground truth versus hallucination) from a single activation vector. Finally, we validate these insights through a geometric consistency check method that effectively detects and corrects these quantization failures during inference. Our code is available at https://github.com/RL-MIND/Shape-of-Addition.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Addition on 10-digit addition 3 terms](<https://www.sota2.com/research/sota/addition-on-10-digit-addition-3-terms.md>): Addition; 10-digit addition 3 terms; Token Accuracy: 89.56.

- [Token-level Addition on Addition T3 (val)](<https://www.sota2.com/research/sota/token-level-addition-on-addition-t3-val.md>): Token-level Addition; Addition T3 (val); Token Accuracy: 89.56.

- [Addition on 12-digit addition 3 terms](<https://www.sota2.com/research/sota/addition-on-12-digit-addition-3-terms.md>): Addition; 12-digit addition 3 terms; Token Accuracy: 94.62.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.03645>)

[PDF](<https://arxiv.org/pdf/2606.03645.pdf>)

[Implementation](<https://github.com/RL-MIND/Shape-of-Addition>)
