# ActVAR: Activating Mixtures of Weights and Tokens for Efficient Visual Autoregressive Generation Source: [ActVAR: Activating Mixtures of Weights and Tokens for Efficient Visual Autoregressive Generation]() Collection index: [llms.txt]() Data updated: 2025-11-19T05:04:29.660Z Authors: Kaixin Zhang, Ruiqing Yang, Yuan Zhang, Shan You, Tao Huang Published: 2025-11-17T02:28:06.000Z ## Abstract Visual Autoregressive (VAR) models enable efficient image generation via next-scale prediction but face escalating computational costs as sequence length grows. Existing static pruning methods degrade performance by permanently removing weights or tokens, disrupting pretrained dependencies. To address this, we propose ActVAR, a dynamic activation framework that introduces dual sparsity across model weights and token sequences to enhance efficiency without sacrificing capacity. ActVAR decomposes feedforward networks (FFNs) into lightweight expert sub-networks and employs a learnable router to dynamically select token-specific expert subsets based on content. Simultaneously, a gated token selector identifies high-update-potential tokens for computation while reconstructing unselected tokens to preserve global context and sequence alignment. Training employs a two-stage knowledge distillation strategy, where the original VAR model supervises the learning of routing and gating policies to align with pretrained knowledge. Experiments on the ImageNet $256\\times 256$ benchmark demonstrate that ActVAR achieves up to $21.2\\%$ FLOPs reduction with minimal performance degradation. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()