# LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning Source: [LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning]() Collection index: [llms.txt]() Data updated: 2026-04-07T05:00:04.607Z Authors: Md Kowsher, Haris Mansoor, Nusrat Jahan Prottasha, Ozlem Garibay, Victor Zhu, Zhengping Ji, Chen Chen Published: 2026-02-01T01:37:34.000Z ## Abstract MoE-PEFT methods combine Mixture of Experts with parameter-efficient fine-tuning for multi-task adaptation, but require separate adapters per expert causing trainable parameters to scale linearly with expert count and limiting applicability to adapter-based architectures. We propose LiME (Lightweight Mixture of Experts), which achieves expert specialization through lightweight modulation rather than adapter replication. Instead of separate adapters, LiME uses a single shared PEFT module and modulates its output with lightweight expert vectors, reducing expert parameters while generalizing to any PEFT method. Notably, LiME introduces zero-parameter routing by leveraging existing frozen and adapted representations eliminating learned router parameters typically required per layer. Theoretically, we prove that (i) more experts preserve more task-relevant information and (ii) modulation approximates full expert-specific PEFT with bounded error. LiME further incorporates n-gram windowed routing and adaptive expert selection (Auto Top-K) based on routing confidence. Experiments on MMT-47, a multimodal multi-task benchmark with 47 tasks spanning text, image, and video, demonstrate that LiME achieves competitive or superior performance while using up to 4x fewer trainable parameters and up to 29% faster training compared to corresponding MoE-PEFT baselines. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Commonsense Reasoning on MMT-47 Commonsense Reasoning](): Commonsense Reasoning; MMT-47 Commonsense Reasoning; Accuracy: 84.98. - [Object Motion and Spatial Reasoning on MMT-47](): Object Motion and Spatial Reasoning; MMT-47 Object Motion & Spatial; Accuracy: 65.41. - [Vision Understanding on MMT-47 Vision Benchmark](): Vision Understanding; MMT-47 Vision Benchmark; Accuracy: 78.12. - [Action Understanding on MMT-47 Action Understanding](): Action Understanding; MMT-47 Action Understanding; Accuracy: 53.39. - [Natural Language Understanding on GLUE (General Accuracy)](): Natural Language Understanding; GLUE; Accuracy: 91.19. - [MMT-47 High Level Reasoning](): High-Level Reasoning; MMT-47 High Level Reasoning; Accuracy: 45.65. - [Image Classification on MMT-47](): Image Classification; MMT-47; Accuracy: 94.61. - [Language Understanding on GLUE (test) (SST-2, QNLI, QQP, CoLA, MRPC, STS-B Subset)](): Language Understanding; GLUE (test); SST-2 Score: 95.99. ## Original sources and implementation [Original paper]() [PDF]()