# Difficulty-Controllable Cloze Question Distractor Generation Source: [Difficulty-Controllable Cloze Question Distractor Generation]() Collection index: [llms.txt]() Data updated: 2026-05-21T05:00:05.413Z Authors: Seokhoon Kang, Yejin Jeon, Seonjeong Hwang, Gary Geunbae Lee Published: 2025-11-03T12:42:25.000Z ## Abstract Multiple-choice cloze questions are commonly used to assess linguistic proficiency and comprehension. However, generating high-quality distractors remains challenging, as existing methods often lack adaptability and control over difficulty levels, and the absence of difficulty-annotated datasets further hinders progress. To address these issues, we propose a novel framework for generating distractors with controllable difficulty by leveraging both data augmentation and a multitask learning strategy. First, to create a high-quality, difficulty-annotated dataset, we introduce a two-way distractor generation process to produce diverse and plausible distractors. These candidates are filtered and then categorized by difficulty using an ensemble QA system. Second, this newly created dataset is used to train a difficulty-controllable generation model via multitask learning. Experimental results demonstrate that our method generates high-quality distractors across difficulty levels and substantially outperforms GPT-4o in aligning distractor difficulty with human perception. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Distractor Generation on CLOTH](): Distractor Generation; Cloth; Hardest Accuracy: 73.25. - [Distractor Generation on CLOTH (test)](): Distractor Generation; Cloth (test); Invalid Ratio (Easy): 0.1. - [Distractor Generation on CLOTH Easy](): Distractor Generation; CLOTH Easy; Invalid Ratio: 0. - [Distractor Generation on CLOTH Hard](): Distractor Generation; CLOTH Hard; Invalid Ratio: 4.2. - [Distractor Generation on CLOTH Easy augmented (test)](): Distractor Generation; CLOTH Easy augmented (test); F1@10: 26.64. - [Distractor Generation on CLOTH Hard Augmented (test)](): Distractor Generation; CLOTH Hard Augmented (test); F1@10: 41.98. - [Multiple Choice Question Distractor Generation on CLOTH (test)](): Multiple Choice Question Distractor Generation; Cloth (test); Chosen Ratio: 9.4. - [Distractor Generation on CLOTH original (test)](): Distractor Generation; CLOTH original (test); F1@10: 13.23. ## Original sources and implementation [Original paper]() [PDF]()