# SignMuon: Communication-Efficient Distributed Muon Optimization

Source: [SignMuon: Communication-Efficient Distributed Muon Optimization](<https://www.sota2.com/research/paper/6a0bee81dbc836b87d82c309>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-20T05:00:05.687Z

Authors: Neel Mishra, Kushagara Trivedi, Pawan Kumar

Published: 2026-05-04T13:29:55.000Z

## Abstract

Distributed training of large neural networks is bottlenecked by full-precision gradient communication and by coordinatewise optimizers that ignore the matrix structure of weight tensors. We propose Sign-Muon, a 1-bit, matrix-aware optimizer that combines majority-vote sign aggregation from signSGD with the polar-step framework of Muon. Each worker forms a Muon-style direction by taking the polar factor of its momentum via a Newton--Schulz iteration, transmits only the entrywise signs, and aggregates by majority vote; an optional local polar step further enforces orthogonality at no extra communication cost.   Under spectral-norm smoothness and bounded-variance stochastic gradients, the spectral-norm normalized sign step yields an $\\mathcal{O}(1/\\sqrt{T})$ nonconvex rate for an $\\ell\_1$-based stationarity measure. With unimodal symmetric noise, majority vote across $M$ workers cuts the stochastic term by $1/\\sqrt{M}$, matching signSGD. In the $\\alpha$-$\\beta$ model, distributed Sign-Muon needs only one integer sum-allreduce per iteration; all orthogonalization is local, giving a $32\\times$ bandwidth reduction over float32 ($4\\times$ for int8).   Across 330 CIFAR-10/ResNet-50 configurations Sign-Muon attains the best validation accuracy (92.15\\%); its 4-GPU majority-vote variant reaches 92.02\\% with 37\\% less training time at matched effective batch. On nanoGPT, Sign-Muon achieves lower perplexity and better anytime performance than other sign-based baselines, with favorable weak-scaling up to 16 GPUs.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-10 (val) (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-val-accuracy.md>): Image Classification; CIFAR-10 (val); Accuracy: 92.15.

- [Image Classification on CIFAR-10 (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-accuracy.md>): Image Classification; CIFAR-10; Accuracy: 83.74.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.16311>)

[PDF](<https://arxiv.org/pdf/2605.16311.pdf>)