# ButterflyViT: 354$\\times$ Expert Compression for Edge Vision Transformers

Source: [ButterflyViT: 354$\\times$ Expert Compression for Edge Vision Transformers](<https://www.sota2.com/research/paper/69afa5812d603e6de3e43a36>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-11T05:00:04.181Z

Authors: Aryan Karmore

Published: 2026-03-06T09:59:55.000Z

## Abstract

Deploying sparse Mixture of Experts(MoE) Vision Transformers remains a challenge due to linear expert memory scaling. Linear memory scaling stores $N$ independent expert weight matrices requiring $\\mathcal{O}(N\_E \\cdot d^2)$ memory, which exceeds edge devices memory budget. Current compression methods like quantization, pruning and low-rank factorization reduce constant factors but leave the scaling bottleneck unresolved. We introduce ButterflyViT, a method that treats experts not as independent weight matrices but as geometric reorientations of a unified shared quantized substrate. Diversity among experts arises from viewing different angles of shared capacity, not from redundant storage. By applying learned rotations to a shared ternary prototype, each expert yields $\\mathcal{O}(d\_{\\text{model}} \\cdot d\_{\\text{ff}} + N\_E \\cdot n\_\\ell \\cdot d)$ memory which is sub-linear in the number of experts. To address the unique challenges of vision, a spatial smoothness regulariser is introduced that penalises routing irregularities between adjacent patch tokens, turning patch correlation into a training signal. Across image classification tasks on CIFAR-100, ButterflyViT achieves 354$\\times$ memory reduction at 64 experts with negligible accuracy loss. ButterflyViT allows multiple experts to fit on edge-constrained devices showing that geometric parameterization breaks linear scaling.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-100 (val)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-val.md>): Image Classification; CIFAR-100 (val); Accuracy: 56.24.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.06746>)

[PDF](<https://arxiv.org/pdf/2603.06746.pdf>)
