# M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection Source: [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection]() Collection index: [llms.txt]() Data updated: 2025-10-26T02:26:06.010Z Authors: Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov Published: 2023-05-24T08:55:11.000Z ## Abstract Large language models (LLMs) have demonstrated remarkable capability to generate fluent responses to a wide variety of user queries. However, this has also raised concerns about the potential misuse of such texts in journalism, education, and academia. In this study, we strive to create automated systems that can detect machine-generated texts and pinpoint potential misuse. We first introduce a large-scale benchmark \\textbf{M4}, which is a multi-generator, multi-domain, and multi-lingual corpus for machine-generated text detection. Through an extensive empirical study of this dataset, we show that it is challenging for detectors to generalize well on instances from unseen domains or LLMs. In such cases, detectors tend to misclassify machine-generated text as human-written. These results show that the problem is far from solved and that there is a lot of room for improvement. We believe that our dataset will enable future research towards more robust approaches to this pressing societal problem. The dataset is available at https://github.com/mbzuai-nlp/M4. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [AI-generated text detection on M4GT-Bench Reddit (test)](): AI-generated text detection; M4GT-Bench Reddit (test); Accuracy: 97.2. - [AI-generated text detection on M4GT-Bench arXiv (test)](): AI-generated text detection; M4GT-Bench arXiv (test); Accuracy: 98.5. - [LLM Detection Interpretability on M4 (test)](): LLM Detection Interpretability; M4 (test); Accuracy (Human Judgements): 57.3. - [Machine-generated text detection on Wikipedia M4 benchmark (test)](): Machine-generated text detection; Wikipedia M4 benchmark (test); AUROC: 97.8. - [Machine-Generated Text Detection on Average M4 benchmark (test)](): Machine-generated text detection; Average M4 benchmark (test); AUROC: 98.1. - [Machine-generated text detection on WikiHow M4 benchmark (test)](): Machine-generated text detection; WikiHow M4 benchmark (test); AUROC: 97.5. ## Original sources and implementation [Original paper]() [PDF]()