# STAR: Rethinking MoE Routing as Structure-Aware Subspace Learning Source: [STAR: Rethinking MoE Routing as Structure-Aware Subspace Learning]() Collection index: [llms.txt]() Data updated: 2026-06-10T05:01:35.776Z Authors: Sumin Park, Noseong Park Published: 2026-06-07T20:07:24.000Z ## Abstract Mixture-of-Experts (MoE) scales model capacity efficiently by selectively routing inputs to a specialized subset of experts. However, input-expert specialization, the core motivation of MoE, critically depends on whether the router is actually aware of input structure. In practice, MoE routing is typically implemented as a shallow linear projection with limited awareness of input representation, which often leads to unstable routing. We propose STAR, a Structure Aware Routing that rethinks MoE routing as a subspace learning problem by augmenting standard learnable routing with an evolving principal subspace that tracks dominant input structure via Generalized Hebbian Algorithm (GHA). By aligning routing decisions directly with input structure, STAR enables stable expert specialization. We evaluate STAR on controlled synthetic setup and large-scale language and vision tasks, where it consistently improves routing quality and downstream performance over strong MoE baselines. Moreover, optional test-time subspace updates further enhance routing robustness and generalization under input distribution shifts. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Natural Language Understanding on GLUE (CoLA, MRPC, QNLI, MNLI, RTE Subset)](): Natural Language Understanding; GLUE; CoLA Score: 66.62. - [Zero-shot Question Answering and Language Modeling on Commonsense and Reading Comprehension Suite (ARC-c, ARC-e, BoolQ, HellaSwag, LAMBADA, PIQA, RACE)](): Zero-shot Question Answering and Language Modeling; Commonsense and Reading Comprehension Suite (ARC-c, ARC-e, BoolQ, HellaSwag, LAMBADA, PIQA, RACE) zero-shot; ARC-c Accuracy: 26.37. - [Image Classification on ImageNet-C 1.0 (Severity 1)](): Image Classification; ImageNet-C 1.0 (Severity 1); Accuracy (Gaussian): 60.67. - [Image Classification on ImageNet-C Severity 5 1.0](): Image Classification; ImageNet-C Severity 5 1.0; Error Rate (Gaussian): 14.8. - [Image Classification on ImageNet-C Severity 3 1.0](): Image Classification; ImageNet-C Severity 3 1.0; Accuracy (Gauss Corruption): 43.76. ## Original sources and implementation [Original paper]() [PDF]()