# DataMUX: Data Multiplexing for Neural Networks

Source: [DataMUX: Data Multiplexing for Neural Networks](<https://www.sota2.com/research/paper/68fa6de91e209131cea3ce20>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-24T17:56:03.603Z

Authors: Vishvak Murahari, Carlos E. Jimenez, Runzhe Yang, Karthik Narasimhan

Published: 2022-02-18T17:35:33.000Z

## Abstract

In this paper, we introduce data multiplexing (DataMUX), a technique that enables deep neural networks to process multiple inputs simultaneously using a single compact representation. DataMUX demonstrates that neural networks are capable of generating accurate predictions over mixtures of inputs, resulting in increased throughput with minimal extra memory requirements. Our approach uses two key components -- 1) a multiplexing layer that performs a fixed linear transformation to each input before combining them to create a mixed representation of the same size as a single input, which is then processed by the base network, and 2) a demultiplexing layer that converts the base network's output back into independent representations before producing predictions for each input. We show the viability of DataMUX for different architectures (Transformers, and to a lesser extent MLPs and CNNs) across six different tasks spanning sentence classification, named entity recognition and image classification. For instance, DataMUX for Transformers can multiplex up to $20$x/$40$x inputs, achieving $11$x/$18$x increase in throughput with minimal absolute performance drops of $\<2\\%$ and $\<4\\%$ respectively on MNLI, a natural language inference task. We also provide a theoretical construction for multiplexing in self-attention networks and analyze the effect of various design elements in DataMUX.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Associative Recall on Synthetic sequence modeling Associative recall](<https://www.sota2.com/research/sota/associative-recall-on-synthetic-sequence-modeling-associative-recall.md>): Associative Recall; Synthetic sequence modeling Associative recall; Accuracy: 20.04.

- [Induction head on Synthetic sequence modeling Induction head](<https://www.sota2.com/research/sota/induction-head-on-synthetic-sequence-modeling-induction-head.md>): Induction head; Synthetic sequence modeling Induction head; Accuracy: 6.06.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2202.09318>)

[PDF](<https://arxiv.org/pdf/2202.09318.pdf>)
