# M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection Source: [M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection]() Collection index: [llms.txt]() Data updated: 2025-10-26T02:57:11.769Z Authors: Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohanned Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov Published: 2024-02-17T02:50:33.000Z ## Abstract The advent of Large Language Models (LLMs) has brought an unprecedented surge in machine-generated text (MGT) across diverse channels. This raises legitimate concerns about its potential misuse and societal implications. The need to identify and differentiate such content from genuine human-generated text is critical in combating disinformation, preserving the integrity of education and scientific fields, and maintaining trust in communication. In this work, we address this problem by introducing a new benchmark based on a multilingual, multi-domain, and multi-generator corpus of MGTs -- M4GT-Bench. The benchmark is compiled of three tasks: (1) mono-lingual and multi-lingual binary MGT detection; (2) multi-way detection where one need to identify, which particular model generated the text; and (3) mixed human-machine text detection, where a word boundary delimiting MGT from human-written content should be determined. On the developed benchmark, we have tested several MGT detection baselines and also conducted an evaluation of human performance. We see that obtaining good performance in MGT detection usually requires an access to the training data from the same domain and generators. The benchmark is available at https://github.com/mbzuai-nlp/M4GT-Bench. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [AI-generated text detection on M4GT-Bench All (test)](): AI-generated text detection; M4GT-Bench All (test); Precision: 96.96. - [AI-generated text detection on M4GT-Bench PeerRead (test)](): AI-generated text detection; M4GT-Bench PeerRead (test); Precision: 70.58. - [AI-generated text detection on M4GT-Bench WikiHow (test)](): AI-generated text detection; M4GT-Bench WikiHow (test); Precision: 72.62. - [AI-generated text detection on M4GT-Bench Wikipedia (test)](): AI-generated text detection; M4GT-Bench Wikipedia (test); Precision: 60.04. - [AI-generated text detection on M4GT-Bench OUTFOX (test)](): AI-generated text detection; M4GT-Bench OUTFOX (test); Precision: 71.71. - [AI-generated text detection on M4GT-Bench Italian (test)](): AI-generated text detection; M4GT-Bench Italian (test); Precision: 98.86. - [AI-generated text detection on M4GT-Bench Chinese (test)](): AI-generated text detection; M4GT-Bench Chinese (test); Precision: 92.71. - [AI-generated text detection on M4GT-Bench English (test)](): AI-generated text detection; M4GT-Bench English (test); Precision: 88.72. - [AI-generated text detection on M4GT-Bench German (test)](): AI-generated text detection; M4GT-Bench German (test); Precision: 78.85. - [AI-generated text detection on M4GT-Bench Russian (test)](): AI-generated text detection; M4GT-Bench Russian (test); Precision: 96.06. - [AI-generated text detection on M4GT-Bench Reddit (test)](): AI-generated text detection; M4GT-Bench Reddit (test); Accuracy: 74.49. - [AI-generated text detection on M4GT-Bench arXiv (test)](): AI-generated text detection; M4GT-Bench arXiv (test); Accuracy: 43.88. ## Original sources and implementation [Original paper]() [PDF]()