# DuoTok: Source-Aware Dual-Track Tokenization for Multi-Track Music Language Modeling

Source: [DuoTok: Source-Aware Dual-Track Tokenization for Multi-Track Music Language Modeling](<https://www.sota2.com/research/paper/692689a2d6b0dd33429304ad>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-03T05:00:04.568Z

Authors: Rui Lin, Zhiyue Wu, Jiahe Le, Kangdi Wang, Weixiong Chen, Junyu Dai, Tao Jiang

Published: 2025-11-25T11:53:57.000Z

## Abstract

Audio tokenization bridges continuous waveforms and multi-track music language models. In dual-track modeling, tokens should preserve three properties at once: high-fidelity reconstruction, strong predictability under a language model, and cross-track correspondence. We introduce DuoTok, a source-aware dual-track tokenizer that addresses this trade-off through staged disentanglement. DuoTok first pretrains a semantic encoder, then regularizes it with multi-task supervision, freezes the encoder, and applies hard dual-codebook routing while keeping auxiliary objectives on quantized codes. A diffusion decoder reconstructs high-frequency details, allowing tokens to focus on structured information for sequence modeling. On standard benchmarks, DuoTok achieves a favorable predictability-fidelity trade-off, reaching the lowest cnBPT while maintaining competitive reconstruction at 0.75 kbps. Under a held-constant dual-track language modeling protocol, enBPT also improves, indicating gains beyond codebook size effects. Controlled diagnostics show larger predictability costs under cross-track corruption and larger gains from longer context, suggesting that models trained on DuoTok tokens use cross-track structure and non-local history.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Track-to-track modeling on Codec Benchmark Vocal to Accompaniment](<https://www.sota2.com/research/sota/track-to-track-modeling-on-codec-benchmark-vocal-to-accompaniment.md>): Track-to-track modeling; Codec Benchmark Vocal to Accompaniment; Accuracy @1: 13.

- [Unconditional multi-track modeling on Multi-track Music Codec Vocal track](<https://www.sota2.com/research/sota/unconditional-multi-track-modeling-on-multi-track-music-codec-vocal-track.md>): Unconditional multi-track modeling; Multi-track Music Codec Vocal track; Accuracy @1: 14.

- [Unconditional multi-track modeling on Multi-track Music Codec Accompaniment track](<https://www.sota2.com/research/sota/unconditional-multi-track-modeling-on-multi-track-music-codec-accompaniment-track.md>): Unconditional multi-track modeling; Multi-track Music Codec Accompaniment track; Accuracy @1: 11.

- [Unconditional multi-track modeling on Multi-track Music Codec Average](<https://www.sota2.com/research/sota/unconditional-multi-track-modeling-on-multi-track-music-codec-average.md>): Unconditional multi-track modeling; Multi-track Music Codec Average; cnBPT: 0.483.

- [Neural Audio Coding on Codec Benchmark](<https://www.sota2.com/research/sota/neural-audio-coding-on-codec-benchmark.md>): Neural Audio Coding; Codec Benchmark; cnBPT: 48.

- [Audio Tagging on MTT](<https://www.sota2.com/research/sota/audio-tagging-on-mtt.md>): Audio Tagging; MTT; MTT AP: 35.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.20224>)

[PDF](<https://arxiv.org/pdf/2511.20224.pdf>)