# mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences

Source: [mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences](<https://www.sota2.com/research/paper/68fa71091e209131cea4ecc1>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:56:42.060Z

Authors: David Uthus, Santiago Onta\\~n\\'on, Joshua Ainslie, Mandy Guo

Published: 2023-05-18T17:22:53.000Z

## Abstract

We present our work on developing a multilingual, efficient text-to-text transformer that is suitable for handling long inputs. This model, called mLongT5, builds upon the architecture of LongT5, while leveraging the multilingual datasets used for pretraining mT5 and the pretraining tasks of UL2. We evaluate this model on a variety of multilingual summarization and question-answering tasks, and the results show stronger performance for mLongT5 when compared to existing multilingual models such as mBART or M-BERT.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2305.11129>)

[PDF](<https://arxiv.org/pdf/2305.11129.pdf>)