# CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model Source: [CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model]() Collection index: [llms.txt]() Data updated: 2025-10-27T06:31:06.558Z Authors: Shang-Hsuan Chiang, Ssu-Cheng Wang, Yao-Chung Fan Published: 2024-03-15T14:14:26.000Z ## Abstract Manually designing cloze test consumes enormous time and efforts. The major challenge lies in wrong option (distractor) selection. Having carefully-design distractors improves the effectiveness of learner ability assessment. As a result, the idea of automatically generating cloze distractor is motivated. In this paper, we investigate cloze distractor generation by exploring the employment of pre-trained language models (PLMs) as an alternative for candidate distractor generation. Experiments show that the PLM-enhanced model brings a substantial performance improvement. Our best performing model advances the state-of-the-art result from 14.94 to 34.17 (NDCG@10 score). Our code and dataset is available at https://github.com/AndyChiangSH/CDGP. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Distractor Generation on CLOTH original (test)](): Distractor Generation; CLOTH original (test); F1@10: 15.37. - [Distractor Generation on MCQ (test)](): Distractor Generation; MCQ (test); P@1: 10.81. ## Original sources and implementation [Original paper]() [PDF]()